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Abstract 

The  Euro-NATO  Joint  Jet  Pilot  Training  (ENJJPT)  program  at  Sheppard  AFB  conducts 
Undergraduate  Pilot  Training  (UPT)  for  13  NATO  nations  with  a  focus  on  producing  premier 
lighter  pilots.  As  ENJJPT  transitions  to  the  new  T-6  Texan  II,  the  leadership  is  examining  if  the 
current  assignment  model  meets  the  needs  of  the  US  Air  Force  for  the  US  students.  To  assign 
US  students,  the  Senior  National  Representative  uses  the  Merit  Assignment  Scoring  System 
(MASS)  to  rank  order  students  and  assign  aircraft  based  on  preference  and  availability  of 
assignments.  MASS  accounts  for  every  activity  in  pilot  training  as  well  as  a  subjective  input 
from  the  instructors  as  to  the  overall  attitude  and  performance  of  the  student.  The  score  obtained 
from  the  MASS  is  categorical  by  assigning  a  weighting  to  a  particular  category  of  performance. 
Currently,  there  is  no  direct  link  between  the  skills  needed  to  fly  modern  fighter  aircraft  and  the 
MASS.  Additionally,  many  of  the  skills  learned  in  pilot  training  span  multiple  categories  and  it 
is  possible  for  a  deficiency  to  be  buried  in  the  MASS  score.  The  goal  of  this  research  was  to 
identify  the  core  skills  required  to  fly  the  various  fighter  aircraft  through  the  use  of  a  Combat  Air 
Forces  (CAF)  wide  survey  instrument,  interviews,  and  working  group  inputs.  An  assignment 
model  was  created  with  a  focus  on  assigning  students  based  on  skill  strengths.  After  the  core 
skills  were  identified  and  related  to  UPT  events,  a  value  hierarchy  was  created  and  a  model 
developed  to  identify  the  best  aircraft  fit  for  a  student  based  on  their  performance  as  related  to 
the  skill  sets.  This  paper  frames  the  issues,  outlines  the  methodology  used  to  define  the  skill  sets, 
and  discusses  the  development  of  the  model.  Finally,  recommendations  are  made  on  future 
changes  to  MASS,  the  UPT  student  assignment  process,  and  the  pilot  training  syllabus. 
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IMPROVING  THE  UPT  STUDENT  FOLLOW-ON 
ASSIGNMENT  SELECTION  PROCESS 

I.  Introduction 

Background 

Near  the  end  of  twelve  intense  and  often  heartbreaking  months  of  training,  a  select  group 
of  military  officers  receive  word  of  their  next  assignment  and  the  aircraft  they  will  fly.  For 
many,  it  is  an  exciting  and  exhilarating  time  as  they  fulfill  their  dreams  of  becoming  a  fighter 
pilot.  For  some,  disappointment  rains  down  as  they  get  their  fourth,  fifth  or  maybe  even  their 
last  choice  aircraft.  And  for  some,  the  excitement  of  assignment  night  lasts  only  a  short  while  as 
their  flying  experience  turns  into  a  nightmare  when  they  fail  to  complete  their  follow-on  training. 

A  few  weeks  after  assignment  night,  these  same  officers  complete  one  of  the  highest 
achievements  of  their  careers  -  obtaining  their  wings.  To  reach  this  point,  they  have  successfully 
completed  the  training  prescribed  in  the  AETC  Undergraduate  Pilot  Training  (UPT)1  syllabus 
including  completing  140  hours  of  flight  in  an  aircraft,  passing  eight  demanding  check  rides  and 
enduring  the  scrutiny  of  experienced  instructors. 

The  assignment  system  of  students  completing  pilot  training  has  changed  relatively  little 
over  the  years  while  fighter  aircraft  have  been  revolutionized.  Leaps  in  technology  have 
transformed  the  cockpits  of  the  aircraft  these  pilots  fly.  Additionally,  the  interface  between  man 
and  machine  is  no  longer  a  simple  gauge  or  switch  but  a  multi-layered  multi-functional  display 
with  Hands-On-Throttle-And-Stick  (HOTAS)  capabilities  challenging  even  the  most  advanced 
students.  Training  aircraft  have  also  improved.  The  T-6  Texan  is  allowing  students  to  learn 
instrument  flying  skills  in  a  similar  glass  cockpit  to  the  F-15E,  F-16  and  F-22  aircraft  they  will 

1  UPT  is  the  generic  term  for  pilot  training  which  includes  UPT,  Specialized  UPT  (SUPT),  Joint 
SUPT  and  ENJJPT. 


1 


fly  in  combat.  The  T-38C  also  has  glass  cockpit  instrumentation  and  challenges  the  students  to 
not  only  fly  a  high  performance  machine  but  to  multi-task  and  interface  like  never  before. 

The  current  assignment  system  uses  the  Merit  Assignment  Scoring  System  (MASS)  which  is 
a  formula  of  weighted  scores  taken  from  graded  events  throughout  training  in  various  categories 
of  training  (see  Table  1).  Students  are  rank-ordered  and  given  assignments  based  on  a  raw  score. 
Although  the  categories  are  weighted,  no  specific  skill  area  is  graded  and  there  is  no  relation  of 
core  skills  needed  in  flying  modem  day  fighters  to  UPT  grading,  evaluation,  and  assignment 
selection.  As  a  result,  the  potential  exists  for  lost  training,  which  equates  to  lost  money,  by 
losing  a  student  (washout)  in  follow-on  training  or  requiring  additional  training  when  a 
deficiency  could  have  possibly  been  identified  at  an  earlier  stage  in  the  pipeline.  Finally,  there  is 
little  or  no  “reach-back”  from  the  fighter  community/units  to  relate  skills  needed  to  actual 
maneuvers  flown  in  pilot  training. 


Table  1  -  Current  ENJJPT  MASS  breakdown 


Flying 

70% 

Category  Check  Maneuver  T-Score 

50% 

T-37 

T-38 

Contact 

(16%) 

Contact 

(24%) 

Advanced  Contact 

(24%) 

Instrument 

(28%) 

Instrument 

(24%) 

Basic  Formation 

(20%) 

Formation 

(20%) 

Advanced  Formation 

(28%) 

Low  Level 

(16%) 

Daily  Performance 

20% 

Flight  Line  Test  T-Score 

(25%) 

Daily  Maneuver  T-Score 

(75%) 

Academics  T-Score 

10% 

Flight  Commander  Ranking  T-Score 

20% 

Total 

100% 
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UPT  still  needs  to  be  the  bedrock  of  basic  skills  but  if  the  assignment  system  selects  the 
student  who  can  best  fly  an  aerobatic  loop  verses  who  has  the  required  skill  to  operate  a  complex 
machine  under  high  G  loading,  students  will  continue  to  have  problems  as  fighter  aircraft 
advance  in  capabilities. 

Euro-NATO  Joint  Jet  Pilot  Training  (ENJJPT) 

The  80th  Flying  Training  Wing  is  located  at  Sheppard  AFB  in  Wichita  Falls,  Texas  and 
was  established  in  1986  as  the  ENJJPT  program  training  student  pilots  from  across  NATO. 
Currently,  thirteen  NATO  countries  provide  monetary  support,  nine  countries  provide  instructor 
pilots,  and  five  countries  have  students  enrolled  in  the  program.  The  Steering  Committee  (SC) 
meets  twice  a  year  to  discuss  policy  and  issues  affecting  ENJJPT  and  decide  on  future  changes  to 
the  syllabus  and  training  program. 

Prior  to  graduating,  the  students  select  which  follow-on  aircraft  they  desire  to  fly.  The 
reasons  the  students  have  for  selecting  an  aircraft  type  vary  and  may  include  historical,  instructor 
influence,  family  or  personal  reasons.  The  MASS  score  is  used  along  with  the  student’s 
preference  to  match  the  individual  to  an  available  training  slot.  At  ENJJPT,  the  US  Senior 
National  Representative  (SNR)  is  the  individual  who  matches  the  US  students  to  available 
assignments.  If  a  student  is  deemed  not  fit  to  fly  a  particular  aircraft  and  no  options  are 
available,  the  US  SNR  will  work  to  trade  the  aircraft  assignment  with  another  training  base.  This 
is  most  frequently  used  when  a  student  is  deemed  not  able  to  fly  fighter  aircraft  and  a  bomber 
assignment  is  needed. 

One  final  component  of  the  assignment  process  is  a  subjective  “Flight  Commander” 
ranking.  The  instructors  who  have  direct  daily  contact  and  fly  with  the  student  provide  input  to 
rank  order  the  students  in  a  class.  This  ranking  results  in  a  subjective  score  which  counts  toward 
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20%  of  the  student’s  overall  MASS  score.  The  instructors  consider  the  effort  and  attitude  the 


student  has  put  forth  in  the  program  and  is  very  subjective.  Although  the  current  ranking  system 
contains  elements  to  prevent  favoritism,  it  can  be  shown  and  has  occurred  where  the  number  one 
pilot  in  terms  of  skill  and  flying  ability  ranks  toward  the  bottom  (10  out  of  13)  of  a  class  in 
overall  MASS  score.  Under  the  current  assignment  process,  this  particular  student  might  not 
receive  his/her  aircraft  preference.  Several  instructors  have  voiced  concern  over  the  ultimate 
impact  the  Flight  Commander  Ranking  has  on  the  student’s  future  assignment  and  career. 

ENJJPT  is  considered  to  be  the  premier  pilot  training  program  in  the  world.  For  US 
students,  most  students  are  assigned  to  fighter  aircraft  upon  graduation.  Students  who  do  not 
meet  a  minimum  level  are  assigned  to  a  bomber  aircraft.  Generally,  there  are  a  limited  number 
of  bomber  assignments  per  class. 

Research  Objective 

The  ultimate  goal  of  AETC  is  to  produce  a  pilot  with  skills  which  have  the  best 
possibility  of  graduating  from  follow-on  programs.  The  objective  of  this  research  effort  is  to  1) 
identify  Core  Competencies  (skill  sets)  required  to  fly  modern  fighter  aircraft  and  2)  to  develop 
an  improved  MASS  using  a  composite  “skills”  score  that  will  better  predict  future  perfonnance. 
The  end  result  is  to  develop  a  relatively  objective  assignment  system  based  on  “skill  sets”,  as 
opposed  to  categorical  perfonnance,  which  the  US  SNR  at  ENJJPT  can  use  to  assist  in 
determining  assignments  for  the  students.  This  system  could  be  used  to  identify  potential 
deficiencies  at  earlier,  less  expensive  stages  of  training  recommending  appropriate  remedial 
actions  to  benefit  both  the  student  and  the  Air  Force.  This  research  provides  such  an  assignment 
system  in  the  fonn  of  an  Excel  based  spreadsheet  model  which  can  be  used  on  a  standard 
desktop  PC. 
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ENJJPT  was  selected  as  the  primary  focus  of  this  research  due  to  my  background  and 
familiarity  with  the  program  as  well  as  the  ENJJPT  ’08  initiative.  As  ENJJPT  replaces  the 
venerable  T-37  with  the  new  T-6  trainer  in  September  of  2008,  the  leadership  and  SC  felt  the 
time  was  appropriate  to  conduct  a  top  to  bottom  review  of  the  program.  AETC  is  also  interested 
in  the  results  and  the  potential  implementation  at  the  remaining  UPT  bases. 

Examples 

Throughout  portions  of  this  paper,  examples  will  be  presented  to  illustrate  the  model  and  the 
concepts  behind  the  new  assignment  process.  The  situations  and  students  are  completely 
fictional  and  any  similarities  to  an  actual  individual  are  purely  coincidental. 

Paper  Outline 

Chapter  2  will  discuss  previous  efforts  to  identify  and  relate  skills  sets  and  discuss  current 
on-going  work  in  the  improvement  of  UPT.  Chapter  3  will  present  the  methodology  used  to 
gather  the  data  and  frame  the  problem.  Chapter  4  is  an  analysis  of  the  data  and  how  it  relates  to 
the  model  development.  Chapter  5  details  the  actual  model  development  and  implementation.  I 
present  recommendations  for  model  use,  changes  to  UPT,  and  modifications  to  current  policy  as 
it  relates  to  the  new  assignment  process  presented  in  Chapter  6.  The  final  portion  of  the  main 
document,  Chapter  7,  provides  a  framework  for  future  and  continued  work  on  this  subject. 
Finally,  the  appendices  contain  the  various  surveys,  tables  of  results  and  skill  definitions  and 
relationships. 
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II.  Past  Research/Literature  Review 


Chapter  Overview 

This  chapter  will  present  previous  research  and  introduce  other  efforts  in  the  area  of 
predictive  performance.  In  preparing  for  this  research,  an  extensive  search  of  literature  was 
conducted  and  several  reviews  of  studies  follow  with  ties  to  UPT  and  understanding  the  nature  of 
student  behavior  as  it  relates  to  future  outcomes  in  follow-on  training. 

The  idea  of  using  current  behavior  as  a  future  predictor  in  the  military  is  not  new  and 
dates  back  to  pre- World  War  I.  Many  examples  can  be  given  showing  research  in  the  area  of 
human  behavior  but  will  not  presented  here.  What  will  be  presented  is  the  literature  which  has 
some  direct  or  indirect  tie  to  UPT  and  student  performance. 

Future  Training  Aircraft 

In  2005,  the  RAND  Corporation  conducted  a  study  “Assessing  the  Impact  of  Future 
Operations  on  Training  Aircraft  Requirements”  (Ausink  and  others,  2005).  The  study  was 
commissioned  under  Gen  Donald  Cook,  then  AETC  Commander,  to  determine  how  the  skills 
required  to  fly  future  Air  Force  aircraft  might  affect  the  decision  to  modify  or  replace  the  current 
fleet  of  training  aircraft.  RAND  attempted  to  answer  the  question  of  what  new  skills,  if  any, 
should  be  taught  in  UPT  to  meet  the  challenges  of  modern  war  fighting  machines. 

The  research  was  conducted  through  interviews  with  students  and  pilots  in  all  stages  of 
training  and  operational  experience.  With  an  open-ended  questionnaire,  they  gathered  feedback 
on  skills  the  pilots  felt  were  best  learned  in  the  operational  aircraft,  in  the  simulator,  and/or  in  a 
training  aircraft. 

During  the  course  of  their  study,  RAND  did  answer  many  issues  concerning  future  skills 
but  the  bottom  line  did  not  formalize  any  particular  “skill  sets.”  The  study  did  provide  several 
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generalized  broad  areas  and  provided  the  following  recommendations  for  the  future  of  UPT 
(Ausink  and  others,  2005 :xv): 

“Collection,  synthesis  and  prioritization  of  information  in  the  cockpit”  will  have  greater 
demands  on  the  pilot 

Flying  and  controlling  the  aircraft  must  continue  to  be  second  nature 
Pilots  will  be  challenged  with  greater  responsibilities  that  are  “focused  on  the 
management  of  information,  sensors,  and  weapons” 

Proficiency  at  “layering  technology  solutions  in  the  cockpit”  must  increase 
Pilot  training  must  continue  to  teach  the  fundamentals  of  flying 

The  report  raised  serious  questions  with  the  F/A-22  and  F-35  and  the  ability  of  a  newly 

minted  pilot  max  performing  these  advanced  aircraft  without  any  experience  with  an  instructor  in 

the  aircraft.  Although  G  Awareness  is  graded  on  many  sorties  throughout  UPT,  it  is  not  a 

specifically  identified  skill.  The  T-38  is  only  capable  of  performing  to  6  Gs  while  the  F/A-22  is 

capable  of  sustaining  9  Gs  -  a  significant  increase.  Since  there  are  no  plans  for  a  2-seat  version 

of  either  platform,  the  pilot  of  these  new  aircraft  must  have  an  innate  skill  of  G  awareness  and  G 

loading.  The  ability  of  a  student  to  recognize  and  perform  while  under  G  loading  is  just  one 

example  of  a  skill  set  which  can  be  defined  and  should  be  used  to  assign  students  to  a  particular 

platform. 

Conclusions  from  the  report  indicate  the  current  fleet  of  training  aircraft  (T-l,  T-6  and 
T-38C)  are  adequate  for  meeting  the  needs  of  training  basic  flying  though.  These  aircraft  are 
acceptable  for  introducing  basic  skills  and  replacement  of  the  training  aircraft  should  be  based 
solely  on  economics  and  training  needs. 

“Concept:  T-38C  Enhanced  Human  Performance  Training” 

AETC/A3  is  currently  researching  “skill-sets”  from  the  perspective  of  human 
performance  (Gillis,  February  2006).  Pilots  appear  to  be  doing  all  the  right  things  under  normal 
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conditions  but  there  is  concern  about  when  a  pilot  faces  adversity.  By  defining  skill-sets,  a 
focused  effort  can  be  directed  toward  training  and  preparing  pilots  for  the  future. 

The  study  focused  on  Task  Management  and  Situational  Awareness  skills.  It  was 
discovered  there  is  a  lack  of  “defined  core  competencies,  training  tasks  and  objectives  related  to 
decision  making,  situational  awareness,  [and]  task  management”  (Gillis,  2006:8),  thus 
identifying  a  key  deficiency  in  the  current  UPT  syllabus.  In  relation  to  follow-on  aircraft 
upgrade  training,  a  lack  of  skill  definition  may  lead  to  a  student  having  to  complete  training  they 
are  not  fully  equipped  to  accomplish  and  may  require  additional  training  or  retraining  at  later 
stages  in  their  upgrade. 


Figure  1  -  Migrating  Skill  Requirements 
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In  the  past,  maneuver  or  equipment-specific  skills  needed  by  the  Combat  Air  Forces 
(CAF)  have  been  brought  back  as  a  requirement  (see  Figure  1).  This  process  does  not 
necessarily  exploit  the  strengths  of  a  particular  training  platform  or  environment.  By  identifying 
a  skill  need,  each  platform  can  “explore  and  leverage  individual  training  media”  (Gillis, 

2006:12). 

The  T-38C  and  T-6  offer  opportunities  to  teach  enhanced  cockpit  task  management.  The 
suggestion  from  the  study  was  to  expand  UPT  to  utilize  the  new  and  upgraded  training  aircraft  to 
their  fullest  potential  which  would  mean  a  shift  from  the  “traditional  maneuver-centric”  ideology 
to  a  more  skills-based  approach.  A  key  point  presented  by  Mr.  Gillis  concerning  the  new  aircraft 
is  “how  we  teach  is  of  equal  importance  to  what  we  teach”  (Gillis,  2006:  9).  The  syllabus  needs 
to  be  clear  on  the  skill  definitions  and  focused  on  the  skills  for  the  instructor  to  properly  mold  a 
student  in  the  correct  fashion. 

The  AETC  study  focused  on  two  areas  of  pilot  training  and  development.  Since  the 
initial  stages  of  this  research  were  conducted  independently  and  parallel  to  AETC,  the  concept  of 
skills  based  training  and  assignment  is  somewhat  validated.  This  research  s  now  part  of  the 
AETC  effort  and  will  expand  on  the  skills-based  approach  to  other  facets  of  UPT. 

“Building  the  Fighter  Pilot” 

In  his  paper  “Building  the  Fighter  Pilot:  Developing  Transferable  Skill  Sets”  (Hannan, 
June  2004),  Capt  Harman  presents  the  current  challenges  in  producing  an  F-16  pilot  capable  of 
handling  the  complex  multi-role  fighter  mission.  At  the  core,  focusing  on  a  specific  mission 
limits  the  overall  skill  set  training  (Hannon,  2004: 1).  He  contends  that  a  solid  training  plan 
should  be  a  balance  encompassing  skill  sets  required  on  all  missions  (Harman,  2004:1). 
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The  UPT  syllabus  was  created  many  years  ago  to  prevent  this  “stove-pipe”  warning 
presented  by  Capt  Hannan.  However,  the  goal  may  be  to  produce  a  well-rounded  pilot  who  has 
the  ability  to  safely  fly  an  aircraft,  there  is  little  or  no  emphasis  on  specific  skills.  The  current 
syllabus  details  levels  of  performance  on  specific  maneuvers  and  although  there  is  thought 
behind  the  maneuvers  and  desired  outcome,  the  actual  skill  relation  is  not  presented. 

A  warning  is  also  presented  in  the  paper  concerning  the  “eye-watering  technological 
transformation”  today’s  upgrading  student  faces  (Hannan,  2004:3).  As  stated  previously, 
HOTAS,  Helmet  Mounted  Sight  and  Fighter  Data  Link  are  just  a  few  examples  of  technology  in 
use  today  and  little  has  changed  in  the  training  program  over  the  years  even  though  the  student  is 
now  pushed  to  the  limits  their  bodies  can  take.  The  question  raised  is,  are  the  basic  skills  taught 
in  UPT  enough? 

Capt  Harman  suggests  developing  “effects-based”  training  plan  that  builds  on  the  basic 
skill  sets  taught  in  UPT  and  Introduction  to  Fighter  Fundamentals  (IFF)  (Harman,  2004:5).  He 
feels  the  basics  of  aircraft  control,  fonnation  and  basic  communication  skills  must  be  second 
nature  and  are  best  taught  in  pilot  training.  Without  specific  feedback  on  these  skill  areas,  the 
UPT  instructor  must  subjectively  determine  if  this  goal  is  being  met.  Identifying  and  quantifying 
skills  required  in  follow-on  training  is  a  requirement  and  should  be  part  of  the  UPT  syllabus. 

T-38  Predictors  Study 

Prior  to  the  RAND  study  on  Training  Aircraft  Requirements,  AETC  Studies  and  Analysis 
Squadron  completed  a  study  to  detennine  if  there  was  a  relationship  between  student 
performance  in  T-37  training  (Phase  II)  and  follow-on  perfonnance  in  T-38s  (Phase  III)  (Hoss, 
November  2002).  Although  the  original  purpose  was  to  determine  if  the  best  pilots  were  selected 
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to  go  to  fighter  training,  the  study  intent  became  a  determination  if  the  best  students  in  SUPT 
were  selected  for  the  T-38  track  (fighter/bomber). 

Among  the  assumptions  AETC/SAS  used,  ENJJPT  was  deemed  a  “good  ‘lab’  case” 
(Hoss,  2005:5)  although  ENJJPT  is  not  a  SUPT  program.  All  ENNJPT  Phase  II  students 
currently  go  on  to  T-38s  in  Phase  III  so  the  success/failure  of  the  full  range  of  abilities  could  be 
considered.  AETC/SAS  also  assumed  the  final  MASS  value  is  the  best  indicator  of  a  good  pilot. 
One  final  assumption  was  no  T-38  washout  case  was  considered. 

The  bottom  line  result  and  recommendation  from  the  study  was  the  T-37  scores  did 
predict  the  student  performance  in  T-38s  and  only  the  top  half  of  the  T-37  perfonners  should  be 
sent  to  T-38s  and  eventually  fighter  aircraft.  However,  with  the  limits  imposed  by  the 
assumptions,  key  data  could  have  been  overlooked.  Many  students  do  not  “click”  until  mid-way 
through  Phase  III.  By  not  considering  the  T-38  wash-out  cases,  a  solid  performer  in  T-37s  may 
have  done  poorly  eventually  washing  out  in  T-38s  -  an  important  data  point. 

Pilot  Selection  Model 

Captain  Ian  Young  explored  the  possibilities  of  using  multivariate  techniques  to  improve 
the  Pilot  Candidate  Selection  Model  (PCSM).  PCSM  is  a  predictor  based  on  three  inputs  as  to 
the  success  or  failure  of  a  student  in  UPT  (Young,  March  2002).  The  concept  is  the  Air  Force 
can  save  money  by  not  sending  those  students  who  are  predicted  to  fail  UPT  based  on  known 
factors.  The  current  PCSM  uses  the  applicants  Air  Force  Officer  Qualification  Test  (AFOQT) 
scores,  Basic  Aptitude  Test  (BAT)  score,  and  age  to  predict  the  outcome  in  UPT. 

Using  Discriminate  Analysis,  Artificial  Neural  Network  structures,  and  Factor  Analysis, 
Capt  Young  developed  a  model  improving  on  the  current  PCSM.  Through  conditioning  of  the 
data,  he  created  a  model  which  better  predicts  the  student  success. 
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Multivariate  analysis  provides  an  opportunity  to  delve  into  a  deeper  mathematical 
modeling  of  predictive  methods.  However,  the  research  conducted  and  presented  here 
approaches  the  performance  prediction  of  a  student  based  on  Decision  Analysis  and  Value 
Focused  Thinking  concepts.  Capt  Young’s  model  is  a  good  example  of  additional  research  that 
can  be  accomplished  to  validate  the  concepts  presented  in  this  paper. 

ENJJPT  ‘08 

The  leadership  at  ENJJPT  was  tasked  by  the  SC  to  examine  the  status  of  training  as  the 
T-38  upgrade  is  completed  and  the  T-6  enters  service  at  Sheppard  in  September  of  2008,  to 
determine  if  there  are  better  training  program  alternatives  and  areas  for  efficiency  increases  and 
improvements.  The  original  working  group  was  chaired  by  Lt  Col  Eric  Bogaards,  Royal 
Netherlands  Air  Force,  and  Maj  Bruce  Dobbins,  USAF. 

The  working  group,  with  participating  nation  inputs,  developed  a  plan  to  modify  all 
phases  of  the  ENJJPT  training  program.  The  conclusion  was  to  “download  flying  skills  and 
knowledge  sets  to  teach  the  necessary  skills  earlier  in  flight  training,  exploit  less  expensive 
aircraft,  [and  use  the]  synergistic  effects  in  follow-on  training”  (Dobbins,  2006:8).  By  modifying 
the  flying  training  conducted  at  Sheppard,  the  goal  is  to  improve  the  final  product  given  to 
follow-on  units  with  a  greater  focus  on  fighter  skills.  Although  the  plan  improves  many  areas 
through  exploitation  of  available  technology  and  program  flow,  the  original  proposal  kept  the 
categorical  focus  intact. 

Decision  Analysis 

Decision  Analysis  (DA)  is  a  systematic,  iterative  process  of  evaluating  and  comparing 
alternatives.  There  are  many  tools  associated  with  DA  -  Value  Hierarchies,  Sensitivity  Analysis 
and  Influence  Diagrams.  A  key  principle  of  DA,  and  modeling  in  general,  is  that  these  tools  are 
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not  necessarily  meant  to  solve  a  problem  but  provide  the  decision  maker  a  greater  understanding 
of  the  problem.  Often  through  the  course  of  model  development  a  greater  understanding  of  not 
only  the  problem  but  possible  alternatives  becomes  clearer. 

A  Value  Hierarchy  is  a  visual  representation  and  organization  of  concepts  and  values 
representing  an  organization  or  problem.  The  higher  the  level  in  the  hierarchy,  the  more  general 
the  objectives  while  lower  levels  become  very  detailed.  For  example,  the  top  level  of  the 
hierarchy  presented  here  is  “Pilot  Ability”  (see  Figure  7).  At  the  lowest  levels,  specific  skills 
such  as  “Heading  Control”  are  represented  (see  Figure  8). 

Sensitivity  Analysis  is  probably  the  most  important  and  useful  tool  in  DA.  It  often 
answers  the  sometimes  difficult  question  of  “what  if’  after  a  model  is  developed  and  is  a  means 
of  determining  “what  makes  a  difference  in  this  decision”  (Clemen  and  Riley,  2001 : 175).  There 
are  a  variety  of  output  results  to  assist  a  decision  maker  or  model  developer  in  determining  how 
various  inputs  affect  the  outcome.  Using  the  results,  changes  and  improvements  to  the  model 
can  be  made.  Additionally,  new  alternatives  and  courses  of  action  may  be  discovered. 

Value  Focused  Thinking 

Value  Focused  Thinking  (VFT)  is  a  specific  application  within  DA.  VFT  is  an  approach 
to  decision  making  by  “first  deciding  what  you  want  and  then  figuring  out  how  to  get  it” 
(Keeney,  1992:4).  The  more  traditional  approach,  alternative-based  method,  is  to  list  available 
alternatives  and  select  the  best  choice.  By  first  considering  what  is  important  before  considering 
the  alternatives,  a  broader  range  of  options  may  become  available  and  the  root  problem  can  be 
better  addressed. 

Values,  by  definition,  are  ideas  and  principles  used  for  evaluating  the  issue  (Keeney, 
1992:6).  Shaping  the  values  before  consideration  of  any  alternatives  will  prevent  any  bias 
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toward  a  particular,  possibly  favorite,  alternative.  Keeney  terms  this  as  “Constraint-Free 
Thinking.”  The  principle  behind  VFT  is  to  evaluate  desirable  alternatives  based  on  what  is 
important  to  the  decision  maker. 

A  key  benefit  of  using  VFT  is  that  the  process  helps  better  define  a  problem.  A  decision 
maker  may  feel  they  fully  understand  a  problem  or  issue.  However,  after  research  begins, 
additional  issues  and  problems  may  surface  and  potentially  completely  redefine  the  problem.  By 
framing  the  problem  and  asking  the  questions  about  what  values  are  important  concerning  the 
problem,  the  wider  range  of  concerns  are  addressed  prior  to  ever  considering  the  possible 
alternatives. 

The  process  for  VFT  is  (Keeney,  1992:49): 

1)  Recognize  a  decision  problem 

2)  Specify  values 

3)  Create  alternatives 

4)  Evaluate  alternatives 

5)  Select  an  alternative 

As  part  of  specifying  values,  a  comparative  weighting  is  conducted.  This  weighting 
becomes  a  numeric  percentage  which  will  be  used  when  comparing  alternatives.  Each 
alternative  will  be  evaluated  under  each  value,  given  a  score,  and  the  weighted  scores  are  added 
together.  The  alternative  with  the  highest  score  is  the  best  choice. 

Not  all  objectives  are  equal  and  may  change  over  the  range  of  possibilities.  The  value  of 
an  objective,  and  thus  the  score,  may  change  as  the  input  increases  or  decreases.  If  there  is  a 
constant  increase  or  decrease  in  value  as  the  objective  input  changes,  the  value  function  is 
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considered  linear.  Value  functions  can  take  on  any  form  and  most  are  non-linear.  Piece-wise 
linear  and  exponential  are  two  types  of  functions  used  in  this  research  and  will  be  presented  later. 

VFT  Example 

The  following  is  simplified  example  of  VFT.  The  idea  presented  does  not  relate  to  UPT 
or  the  assignment  process  but  many  of  the  same  concepts  will  be  used  later  in  this  research.  For 
this  example,  a  decision  maker  is  faced  with  choosing  a  restaurant  to  visit  for  lunch.  Before 
considering  the  restaurants  available,  the  VFT  process  looks  at  defining  what  is  important  in  this 
decision.  Distance,  food  selection,  and  price  may  be  three  important  values  to  the  decision 
maker.  There  are  potentially  limitless  possibilities  but  considering  these  three  values  will 
illustrate  the  concept.  Our  decision  maker  is  a  college  student  with  limited  funds  and  must  walk 
to  the  restaurant  so  he  values  price  and  distance  much  higher  than  food  selection.  The  following 
diagram  shows  the  Value  Hierarchy  for  this  decision  and  shows  the  weightings  our  decision 
maker  has  attached  to  each  objective. 


Figure  2  -  VFT  Example  Value  Hierarchy 

Once  each  objective  has  been  weighted,  the  individual  objective  must  be  considered.  For 
distance,  our  decision  maker  must  walk  and  is  willing  to  walk  up  to  1  mile  to  eat.  However,  for 
distances  beyond  1  mile,  the  decision  maker,  although  willing  to  consider  these  restaurants,  is 
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less  receptive.  Thus,  the  value  for  restaurants  greater  than  1  mile  drops  off.  A  piece-wise  linear 
value  function  (see  Figure  3)  represents  the  decision  makers  feelings  concerning  distance. 


Figure  3  -  VFT  example  Distance  Piece-wise  Linear  Function 

For  price,  our  decision  maker  has  concerns  the  greater  the  cost.  In  this  case,  a  decreasing 
exponential  function  is  used.  Rho  is  the  risk  value  the  decision  maker  is  willing  to  accept  and 
mathematically  determines  the  extent  and  direction  of  the  exponential  curve.  A  negative  rho,  in 
this  case,  indicates  risk  averse  and  produces  a  concave  curve.  A  positive  rho  indicates  risk 
seeking  and  results  in  a  convex  curve.  The  value  of  a  decreasing  exponential  function  is: 

~( Hi-Xa ) 

r(x„)=  ‘  A„,L,r  d) 

l-e  p 

V ( Xa )  =  returned  value 
Xa  =  Objective  input  score 
p  =  rho  =  risk  level 

Note:  for  exp  increasing  functions,  the  expression  (Hi -Xa)  is  replaced  by  (Xa  -  Low)  . 


The  value  function  for  food  selection  is  determined  in  a  similar  fashion  by  obtaining 
feedback  from  the  decision  maker.  For  our  example,  the  food  score  will  be  considered  equal  for 
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the  two  competing  restaurants.  In  all  three  objectives,  the  value  of  the  function  changes  over  the 
range  of  objective  inputs  and  can  be  used  to  determine  the  overall  alternative  value.  The  next 
step  is  scoring  of  alternatives.  For  each  restaurant,  a  score  is  calculated  for  distance,  price,  and 
food  selection  based  on  the  value  functions. 


Price  Value  Function 


Figure  4  -  VFT  example  Price  Value  Function 

In  our  example,  restaurant  A  is  A  mile  away  and  the  meal  will  cost  $9.  Restaurant  B  is 
1.5  miles  away  but  will  only  cost  $4.  From  the  above  value  functions,  Restaurant  A  has  scores 
of  .9  and  .4  while  Restaurant  B  has  scores  of  .3  and  .65.  Applying  the  value  hierarchy 
weightings,  restaurant  A  scores  .9*. 25  +  .3 *.6  =  .405  and  restaurant  B  scores  .3 *.25  +  .65 *.6  = 
.465.  Although  restaurant  B  is  further  away,  the  price  of  the  meal  weighs  heavier  and  is  the  best 
alternative. 

As  stated  previously,  DA  and  VFT  are  tools  to  assist  the  decision  maker.  A  choice  may 
be  suggested  by  the  data  but  the  decision  maker  still  has  input  and  can  decide  to  go  elsewhere. 
This  is  an  important  concept  since  no  model  can  perfectly  predict  an  eventual  outcome.  Expert 
opinion  should  consider  and  weigh  the  output  carefully  before  making  an  ultimate  decision. 
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III.  Methodology  /Data  Collection 


Overview 

In  April  05,  the  90th  FTS  conducted  a  small  working  group  examining  the  MASS  and  the 
relationships  found  in  Table  1.  The  proceedings  from  this  group  became  the  base-line  and  seed 
for  this  research.  This  chapter  will  review  the  progression  of  obtaining  and  processing  the  initial 
data  and  present  the  follow-up  data  collection.  The  infonnation  for  the  model  will  be  presented 
but  the  actual  model  development  process  and  methodology  will  be  presented  in  Chapter  V. 

The  initial  step  in  design  and  development  started  with  identification  of  the  core  skills 
required  to  fly  fighter  aircraft  through  a  survey  instrument  sent  to  all  current  fighter  pilots  in  the 
CAF.  These  skills  were  then  related  to  UPT  instructed  and  evaluated  events  during  a  working 
group  session  held  at  Sheppard.  Throughout  the  process,  Decision  Analysis  techniques  were 
used  to  create  and  modify  a  working  mathematical  model  to  assign  students  based  on  their 
strengths  and  the  skills  required  to  fly  each  aircraft  type  based  on  community  inputs  by  current 
pilots  flying  the  particular  Major  Weapons  System  (MWS).  Finally,  model  validation  was 
planned  by  using  survey  responses  from  student’s  instructors  and  relating  these  results  to  the 
student’s  UPT  records.  Unfortunately,  approval  to  collect  the  student  data  was  pending  at  the 
completion  of  this  study  leaving  this  validation  phase  for  future  work. 

The  entire  UPT  training  philosophy  was  examined  to  determine  if  the  current  grading  and 
scoring  scheme  is  appropriately  set-up  for  use  in  the  proposed  assignment  model.  As  part  of  the 
review  and  model  development,  potential  changes  in  the  curriculum  and  syllabus  were  identified. 
These  training  and  grading  methodology  recommendations  will  be  presented  in  Chapter  VI. 

During  the  review  of  many  of  the  skill  areas,  an  assessment  was  made  as  to  whether  the 
skill  is  currently  being  instructed  and  properly  evaluated  in  UPT.  If  an  area  was  identified  as 
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having  no  related  UPT  event,  a  recommendation  is  made  on  how  best  to  develop  the  skill  set. 
These  recommendations  are  combined  with  the  survey  feedback  to  provide  overall  input  to 
AETC  and  ENJJPT  leadership. 

Independent  Review  Board  (IRB)/Survey  Control  Number  (SCN) 

Approval  to  conduct  research  and  distribute  surveys  was  submitted  to  and  obtained  from 
AFPC/DPSAS.  Additionally,  because  human  subjects  were  involved  in  the  research  process, 
approval  from  the  Wright-State  IRB  was  submitted  and  partially  obtained.  Additional  approval 
was  needed  from  the  IRB  and  the  base  legal  office  concerning  research  into  the  students  UPT 
records.  Coordination  and  approval  was  also  obtained  from  AETC/DOZ  to  access  the  students 
UPT  records  in  the  AETC  master  database.  As  stated  above,  this  additional  approval  from  the 
IRB  was  not  obtained  prior  to  completion  of  this  report  and  additional  validation  will  be  a  topic 
for  future  research. 

Fighter  Core  Skills  Survey  (FCSS) 

The  basic  survey  (see  Appendix  A  for  the  full  survey)  was  developed  from  the 
perspective  of  querying  fighter  pilots  on  missions  and  skills  needed  to  fly  their  particular  MWS. 
The  goal  was  to  identify  the  core  skills  needed  to  fly  fighter  aircraft  and  if  possible,  tie  specific 
skill  sets  to  individual  aircraft  types.  Although  the  “missions  flown”  information  gathered  was 
not  critical  to  the  overall  research,  it  provided  a  frame  of  reference  when  relating  skill  sets.  The 
pilots  were  also  asked  to  reflect  on  their  UPT  experience  and  considering  their  current 
experience  and  required  upgrade  training,  recommend  changes  for  UPT  to  better  prepare 
tomorrows  students  for  fighter  training.  The  initial  survey  was  open  ended  and  administered 
only  to  the  fighter  pilot  population  at  AFIT. 
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The  mission  breakdown  and  skills  questions  resulted  in  a  list  of  23  skill  areas  which  were 
then  used  for  the  larger,  CAF-wide  FCSS.  Appendix  A  Screen  5  shows  the  skill  areas  the 
respondent  was  asked  to  rank  order  for  each  mission  they  flew  in  their  MWS.  These  responses 
provided  direct  feedback  on  the  skills  required  for  each  aircraft  type. 

For  each  of  the  23  skill  areas,  the  total  number  of  responses  within  an  aircraft  group  was 
compared.  Using  a  relative  weighting,  a  percentage  score  for  the  skill  area  was  obtained.  Since 
only  nine  specific  skill  areas  are  used  in  the  aircraft  assignment  best-fit  determination,  these  were 
the  only  skill  areas  considered  from  the  FCSS. 

Appendix  A  Screen  6  of  the  FCSS  shows  the  classification  of  the  original  three  broad 
areas.  The  respondent  was  asked  to  rate  each  broad  skill  area  from  1  to  10.  These  responses 
were  meant  to  define  the  weightings  for  the  value  functions  used  in  the  model.  In  theory,  each 
individual  fighter  community  would  value  the  broad  skill  areas  differently  and  their  individual 
preferences  could  be  applied  to  the  model.  The  end  model  diverged  from  using  the  Basic, 
Instrument,  and  Mission  areas.  As  a  result,  a  follow-up  survey  was  required  to  request  opinions 
on  the  new  skill  areas. 

Sheppard  Surveys 

In  preparation  for  the  FCSS,  several  smaller  requests  for  information  were  sent  to  the 
instructor  cadre  at  Sheppard  (see  Appendix  B,  Appendix  C,  and  Appendix  D).  The  objective 
was  to  identify  and  develop  a  value  hierarchy  and  refine  the  problem  statement.  These  surveys 
also  provided  an  opportunity  to  begin  relating  UPT  events  to  skill  areas  and  lay  the  ground  work 
for  the  ENJJPT  MASS  Working  Group  held  in  April  2006. 

As  with  any  model  development,  the  short  surveys  became  a  tool  for  framing  the  issue 
and  understanding  in  greater  detail  how  the  various  aspects  and  pieces  of  information  would 
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eventually  be  brought  together.  The  surveys  also  provided  the  initial  input  for  the  value 
hierarchy  weightings  and  comparison  between  skill  sets. 

Post  ENJJPT  Working  Group,  a  follow-up  survey  (see  Appendix  E)  was  distributed 
among  ENJJPT  instructors  and  the  AFIT  fighter  pilot  population.  This  survey  requested  a 
weighting  opinion  on  the  revised  skill  set  areas.  As  part  of  the  survey,  a  clear  definition  was 
provided  for  each  area  to  ensure  the  respondent  clearly  understood  the  composition  of  the  skill 
set.  A  statistical  analysis  was  conducted  on  the  collected  data  and  an  average  for  each  sub-skill 
area  was  obtained  and  used  for  weighting  within  the  model.  Specifically,  all  responses 
regardless  of  aircraft  background  were  used  in  the  General  Skill  sets  and  Other  Skill  sets.  For 
the  Aircraft  Specific  Skills,  the  results  from  all  the  respondents  with  a  similar  MWS  background 
were  compared  and  used. 

ENJJPT  MASS  Working  Group 

The  intention  of  the  ENJJPT  MASS  Working  Group  was  to  refine  the  value  hierarchy, 
obtain  feedback  on  the  research  effort,  and  relate  UPT  events  to  the  various  skill  areas.  The 
group  was  comprised  of  representatives  from  every  combat  airframe  and  both  training  aircraft. 
Open  discussion  and  direct  input  was  encouraged  and  as  a  result  of  the  group  dialogue,  the 
model  focus  changed. 

The  agenda  found  in  Appendix  I  provided  a  framework  for  the  group.  After  presenting 
the  problem  statement,  a  brainstonning  session  was  used  to  gather  inputs  on  what  qualities  make 
up  a  good  pilot.  The  objective  was  to  create  a  list  of  skills  which  could  be  used  in  the 
assignment  model.  From  these  qualitative  skill  lists,  a  relationship  and  link  between  skill  areas 
and  UPT  events  could  then  be  formed. 
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There  are  varying  opinions  concerning  the  value  of  T-37/T-6  training  versus  T-38 
training  and  whether  check  rides  or  long  tenn  performance  is  a  better  indicator  of  student 
performance.  This  dilemma  was  presented  to  the  group  for  discussion.  Using  a  1000  poker  chip 
technique,  the  group  was  asked  to  collectively  determine  the  relationship  between  each  of  the 
four  areas.  This  visual  technique  involves  placing  poker  chips  (or  any  other  coin  type  object)  in 
front  of  the  decision  maker.  The  group  then  moves  chips  from  one  pile  to  another,  balancing  the 
piles  of  chips  until  the  proper  relative  groupings  are  found.  The  chips  are  counted  and  a 
percentage  of  the  whole  is  determined  for  each  area. 

The  final  working  group  session  involved  another  brainstorming  session  to  relate  UPT 
events  to  the  previously  obtained  skill  areas.  The  group  members  were  asked  to  not  only 
consider  current  UPT  events  but  to  also  introduce  new  areas  for  instruction.  Again,  these  results 
were  qualitative  but  would  provide  a  framework  for  assigning  UPT  graded  events  to  a  skill  area 
during  the  model  development. 

Decision  Analysis 

The  process  of  model  development  began  with  consolidation  of  the  various  survey, 
working  group,  and  individual  inputs.  The  value  hierarchy  was  built,  refined,  and  rebuilt  based 
on  the  inputs.  With  the  hierarchy  framework  developed,  the  weightings  for  each  skill  area  were 
determined  and  recorded  for  use  in  the  model. 

To  develop  the  weightings,  a  review  of  the  missions  and  skill  areas  was  conducted.  Each 
time  a  skill  was  required  to  conduct  a  mission,  it  was  counted  toward  the  whole  for  the  aircraft 
type.  Once  the  total  number  of  skill  areas  were  tabulated  for  each  aircraft,  a  relative  value  was 
calculated  and  a  corresponding  weight  given.  For  each  mission  type,  the  respondent  was  asked 
to  rank  order  the  skills  from  1  to  4  and  provide  any  additional  skill  areas  required  for  each 
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mission  type.  Because  the  responses  did  not  have  a  relative  weighting  attached,  only  the  raw 
numbers  of  each  type  of  skill  were  used  in  the  analysis. 

Student  Evaluation 

To  validate  the  accuracy  of  the  model,  student  surveys  were  developed.  The  initial  intent 
was  to  distribute  these  surveys  to  the  instructor  pilots  of  students  currently  completing  fighter 
aircraft  training.  The  instructors  would  be  asked  to  rate  the  student’s  abilities  in  the  skill  areas 
used  to  define  the  assignment  best  fit  in  the  model  (see  Appendix  H).  After  receiving  the 
responses,  the  UPT  records  of  the  same  student  would  be  obtained  from  the  AETC  database.  For 
each  of  the  nine  sub-skill  areas  in  the  Aircraft  Specific  Skill  set,  the  individual  event  grades 
would  be  recorded  and  a  comparison  between  the  actual  UPT  grades  and  RTU  performance 
would  be  conducted.  With  the  understanding  that  some  students  leam  at  different  rates,  a 
statistical  evaluation  would  be  completed  and  any  modification  or  shifting  of  the  model 
equations  would  be  done.  As  stated  previously,  this  phase  was  not  completed  prior  to 
completion  of  this  study  pending  approval  from  the  legal  office  and  IRB. 
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IV.  Data  Collection  Results  and  Analysis 


Summary 

This  chapter  discusses  the  data  results  and  the  impact  on  the  approach  to  the  remainder  of 
the  project.  The  results  from  the  FCSS  are  discussed  with  specifics  concerning  each  aircraft 
community  presented.  Finally,  the  ENJJPT  MASS  Working  Group  ideas  are  presented. 

General  Results 

The  original  concept  was  to  break  Pilot  Ability  into  three  broad  skill  areas  and  sub¬ 
classify  required  skills  as  shown  in  Figure  5.  However,  from  the  results  of  the  FCSS  and  the 
ENJJPT  MASS  Working  group,  the  broad  areas  began  taking  a  similar  shape  to  the  existing 
assignment  system  using  categorical  performance.  The  survey  results  also  indicated  many 
common  requirements  with  a  small  number  of  specific  of  skill  sets  varying  between  aircraft 
types.  These  two  results  forced  a  shift  in  how  the  skills  were  classified. 


Figure  5-  Original  Broad  Area  Skills  breakdown 
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FCSS  responses 

The  FCSS  was  distributed  to  all  of  the  CONUS  fighter  bases.  Specific  base  information 
was  neither  requested  nor  is  there  any  value  added.  However,  the  results  did  show  a 
disproportionate  amount  of  respondents  in  one  fighter  community  and  lack  of  responses  from 
another.  A  limiting  factor  in  the  survey  responses  was  the  lack  of  information  from  training  and 
bomber  platforms.  Although  the  survey  was  intended  for  the  fighter  community,  infonnation 
concerning  these  other  platfonns  would  have  been  beneficial  and  is  an  area  for  further  research. 

A  total  of  124  surveys  were  received  from  across  AFIT,  Sheppard  and  the  CAF. 
Appendix  F  outlines  the  data  collected.  For  each  of  the  skill  set  areas,  a  breakdown  by  aircraft 
community  was  tabulated.  Additionally,  for  each  data  point,  the  bottom  rows  indicate  the 
average  for  all  the  surveys  collected.  The  follow-up  Updated  Skills  survey  results  are  also 
included  to  complete  the  percentage  breakdown.  A  95%  confidence  interval  for  each  statistical 
point  is  shown  to  indicate  the  potential  error  for  the  average  in  each  category. 

The  table  of  Aircraft  Specific  Skills  displays  the  response  from  each  community 
concerning  sub-skill  areas.  Since  not  all  “required”  skills  are  listed,  the  values  were  proposed  to 
representatives  from  each  community  and  refined  with  the  FCSS  responses  used  as  the  base-line. 
For  example,  the  F-16  community  had  a  0%  response  for  the  Map  Reading  skill.  This  situation 
most  likely  resulted  from  the  Map  Reading  skill  not  being  a  “top  4”  skill  for  any  particular 
mission  but  overall  it  is  a  necessary  skill.  Consulting  the  F-16  representative,  this  value  was 
adjusted  to  approximately  5%  to  show  a  basic  skill  need  but  still  honor  the  community  desire  for 
this  skill  to  not  be  a  major  deciding  factor  in  a  student  being  assigned  to  the  F-16. 
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Broad  Area  Comparisons 

The  initial  hypothesis  was  the  broad  skill  areas  for  each  aircraft  type  would  differ 
significantly  enough  to  be  useful  in  the  assignment  model.  This  became  the  first  area  of  analysis 
from  the  FCSS.  Each  broad  area  was  compared  between  aircraft  types  utilizing  an  ANOVA 
using  the  MINITAB  statistical  program  (see  Figure  6).  For  the  Basic  and  Instrument  broad  area 
skills,  the  resultant  P-value  indicated  the  means  did  not  vary  significantly  between  aircraft  types. 
With  the  Mission  skills,  the  P-value  was  0.000  indicating  a  difference  at  greater  than  a  99%  level 
of  significance.  However,  with  the  T-38  not  having  dedicated  “mission”  skills,  the  data  from 
these  respondents  skewed  the  results  and  as  such,  was  removed.  When  the  T-38  data  was 
removed,  the  P-value  increased  to  0.202  which  was  a  better  indicator  of  no  statistical  difference 


between  any  of  the  aircraft. 


One-way  ANOVA:  Basic  Skills  versus  A/C 

Source  DF  SS  MS  F  P 

A/C  5  3.85  0.77  0.64  0.669 

Error  114  137.14  1.20 

Total  119  140.99 


One-way  ANOVA:  Mission  Skills  versus  A/C 

Source  DF  SS  MS  F  P 

A/C  5  34.862  6.972  11.64  0.000 

Error  114  68.305  0.599 

Total  119  103.167 


S  =  1.097  R-Sq  =  2.73%  R-Sq(adj)  =  0.00% 

Individual  95%  CIs  For  Mean  Based  on  Pooled  StDev 

Level  N  Mean  StDev  - + - + - + - + 

A- 10  2  9.000  1.414  ( - * - ) 

F-15C  65  9.062  1.144  (  — *-) 

F-15E  15  8.667  1.047  ( - * - ) 

F-16  19  9.053  1.026  ( - * - ) 

F-22  16  9.188  0.750  ( - * - ) 

T-38  3  8.333  2.082  ( - * - ) 


11.0 


8.0  9.0  10.0 


One-way  ANOVA:  Instrument  Skills  versus  A/C 

Source  DF  SS  MS  F  P 

A/C  5  17.78  3.56  0.98  0.430 

Error  114  411.81  3.61 

Total  119  429.59 


S  =  1.901  R-Sq  =  4.14%  R-Sq(adj)  =  0.00% 

Individual  95%  CIs  For  Mean  Based  on  Pooled  StDev 


Level 

N 

Mean 

StDev 

- + - + - 

A- 10 

2 

8.500 

0.707 

( - 

- * - 

F-15C 

65 

7.508 

2.070 

F-15E 

15 

6.733 

1.580 

( - 

—  * - ) 

F-16 

19 

7.842 

1.893 

( - * - ) 

F-22 

16 

7.938 

1.482 

( - * - } 

T-38 

3 

8.333 

1.528 

(- 

- * - 

6.0  7.5 


9.0  10.5 


o 

II 

CO 

.7741 

R-Sq 

=  33.79%  R-Sq(adj)  =  30. 

.89% 

Individual  95%  CIs  For  Mean  Based 

on  Pooled  StDev 

Level 

N 

Mean 

A- 10 

2 

10.000 

0.000 

( - *___ 

F-15C 

65 

9.708 

0.605 

(-*) 

F-15E 

15 

9.667 

0.488 

(-*  — ) 

F-16 

19 

9.368 

0.895 

(-*  — ) 

F-22 

16 

9.813 

0.403 

(-*  — ) 

T-38 

3 

6.333 

3.512  ( - * - ) 

6.0  7.5  9.0  10.5 


One-way  ANOVA:  Mission  Skills  versus  A/C  (T-38  removed) 

Source  DF  SS  MS  F  P 

A/C  4  2.362  0.590  1.52  0.202 

Error  112  43.638  0.390 

Total  116  46.000 


S  =  0. 

.6242 

R-Sq 

=  5.13% 

R-Sq(adj)  =  1.75% 

Individual  95% 

CIs  For  Mean  Based  on  Pooled  StDev 

Level 

N 

Mean 

StDev 

- + - + - + - +- 

A- 10 

2 

10.000 

0.000 

( - * - 

F-15C 

65 

9.708 

0.605 

F-15E 

15 

9.667 

0.488 

( - * - } 

F-16 

19 

9.368 

0.895 

( - * - ) 

F-22 

16 

9.813 

0.403 

( - * - } 

9.50  10.00  10.50 


Figure  6  -  ANOVA  results 
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Based  on  this  result  and  continuing  with  a  “skills”  based  approach,  a  new  hierarchy  was 
developed  and  is  shown  in  Figure  7.  Three  broad  areas  are  considered:  General  Skills,  Aircraft 
Specific  Skills,  and  Other  Skills.  The  General  Skills  are  the  foundation  for  which  all  pilots, 
fighter  or  otherwise,  must  have  solid  abilities.  This  foundation  can  be  built  upon  throughout  the 
remainder  of  pilot  training  and  follow-on  fighter  training.  The  Specific  Skills  set  may  be 
applicable  to  each  aircraft  but  carry  a  different  weight  for  each  community.  The  Other  Skills  are 
those  higher  level  or  less  tangible  skills  which  bring  a  pilot  to  the  next  level  of  competence. 


Other  Skills 


Aircraft  Specific  Skills 


Figure  7  -  Broad  Area  Skills  sets 

The  FCSS  results  indicated  three  major  common  areas  -  SA,  Multi-Tasking,  and  Sensor 
Operations.  For  each  individual  aircraft  type,  three  to  four  areas  stood  out  as  possible  specific 
skills.  From  this,  nine  skill  areas  were  identified  and  were  classified  as  the  Aircraft  Specific 
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Skills  set.  Each  of  the  broad  areas  has  been  broken  into  sub-skill  areas  and  these  skills  determine 


the  overall  pilot  ability  and  can  be  linked  to  instructed  UPT  events. 

Figure  8  shows  the  developed  hierarchy.  The  functional  objective  (top  of  the  hierarchy) 
is  the  Pilot  Ability.  Tier  1  shows  the  three  broad  areas  and  tier  2  the  subset  skill  areas. 


Pilot  Ability 


General  Skills 

Aircraft  Specific 

Skills 

Other  Skills 

Situational 
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Mutil-Tasking/ 
Task  Management 
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Interpretation 
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Attitude 
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Figure  8  -  Skill  Subset  breakdown 

For  assignment  purposes  and  determining  the  best  fit  aircraft  then,  the  Aircraft  Specific 
Skills  are  the  key  factors  in  the  model.  For  example,  the  Target  Identification  (TGT  ID)  skill  is 
ranked  very  high  among  bomb  dropping  communities  (A- 10,  F-15E,  F-16)  while  platforms 
geared  more  to  an  air-to-air  role  do  not  have  much  need  for  such  a  skill.  If  a  student  is  strong  in 
TGT  ID,  his/her  best  fit  aircraft  should  be  a  bomb  dropping  platform.  For  each  student  all  skill 
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areas  are  considered  for  overall  ranking  between  students  and  to  differentiate  closely  skilled 
students  when  a  limited  number  of  aircraft  are  available. 

Sortie  Type  Comparison 

For  each  category  of  training,  there  are  four  sortie  classifications  a  student  can  fly.  A 
maneuver  flown  within  a  skill  area  is  further  broken  into  Phase  II  (T-6/T-37)  and  Phase  III 
(T-38)  and  subdivided  between  daily  rides  and  check  rides.  Figure  9  shows  the  breakdown 
between  the  event  classifications.  The  local  weightings  shown  are  a  comparison  of  values 
between  elements  in  a  branch  at  the  same  tier  level.  The  global  weightings  are  a  relative 
weighting  between  all  measurable  values  in  the  hierarchy  on  the  same  tier. 


Figure  9  -  Sortie  Classification  breakdown 

Local  weighting 
(global  weighting) 

The  check  ride  in  each  category  of  training  is  an  important  event  and  determination  of 
student  progression.  The  intent  is  to  place  a  student  under  higher  than  normal  stress  and  evaluate 
if  the  student  has  achieved  the  desired  training  goals  for  the  particular  point  in  the  program. 
Currently,  the  check  ride  scores  are  weighted  much  higher  than  the  daily  scores  (see  Table  1). 
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However,  feedback  obtained  from  the  CAF  and  the  instructors  from  the  ENJJPT  Working  Group 
indicated  the  emphasis  should  be  placed  on  the  long  run  daily  performance  of  a  student.  The 
result  was  a  shift  to  heavier  daily  ride  weightings  and  since  ENJJPT  is  a  fighter  focused  program, 
the  higher  performance  phase  of  T-38  training  was  favored  over  Phase  II  perfonnance. 

Student  Preference 

A  final  portion  of  the  assignment  equation  is  the  student’s  aircraft  preference  and  desires. 
A  student  is  partially,  if  not  completely,  motivated  by  the  idea  of  flying  a  particular  aircraft  and 
performing  a  certain  mission.  Adding  a  value  in  the  model  to  represent  this  preference 
acknowledges  this  motivation. 

From  survey  and  working  group  feedback,  it  was  seen  that  a  student  in  the  top  1/3  of 
his/her  skill  group  should  not  have  their  assignment  swayed  by  performance  but  more  by  aircraft 
preference.  To  achieve  this  affect,  each  of  the  skill  value  functions  is  designed  to  have  the 
highest  value  returned  at  2/3  of  the  overall  score  potential.  Any  score  above  the  2/3  point,  the 
student  will  receive  no  additional  points  in  the  particular  skill  set.  Again,  validation  with  the 
actual  student  data  may  result  in  adjustments  to  these  curves  and  changes  to  this  1/3-2/3  plan. 
Value  Functions 

The  value  functions  for  the  skill  areas  and  student  preference  were  obtained  through 
interviews  and  opinions  obtained  from  both  CAF  pilots  and  training  instructors.  The  scores  are 
somewhat  arbitrary  since  the  complete  database  table  has  not  been  completed.  However,  the 
concepts  are  the  same  regardless  of  the  scores  used  and  are  easily  modified  in  the  model. 

All  three  of  the  broad  skill  sets  (General  -  Figure  10,  Other  -  Figure  11,  and  Specific  - 
Figure  12)  are  represented  by  linear  functions  for  the  first  2/3  of  the  score  points  and  then  by  a 
constant  value  of  1  for  the  top  1/3.  When  combined  with  the  Student  Preference  value 
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(Figure  13),  a  student  in  the  top  1/3  of  the  skill  area  will  have  the  Student  Preference  as  the 
leading  factor  in  the  best  fit  equation.  A  linear  function  was  primary  used  as  a  base-line  case  due 
to  its  simplicity  and  clarity  to  understand  the  function.  The  “Value  Breakdown”  page  of  the 
Excel  spreadsheet  model  has  the  capability  to  input  parameters  for  an  increasing  exponential 
value  function.  During  validation,  if  it  is  detennined  the  General  and  Other  Skill  areas  do  vary 
between  aircraft  types,  the  Specific  Skills  value  function  can  be  modified  to  continue  in  an 
increasing  fashion  for  the  top  1/3  of  the  skill  score,  while  General  and  Other  remain  the  same, 
acknowledging  the  Specific  Skills  do  play  a  greater  role  in  the  ability  of  the  student. 


General  Skills  Value  Function 


Score 


Figure  10  -  General  Skills  Value  Function 


Other  Skills  Value  Function 


Figure  11  -  Other  Skills  Value  Function 


31 


Figure  12  -  Specific  Skills  Value  Function 

With  eight  distinct  points  representing  the  student’s  aircraft  preference,  a  piece-wise 
linear  function  is  the  best  depiction  of  value.  As  can  be  seen  in  Figure  13,  only  a  slight  decrease 
exists  for  the  first  three  aircraft  choices  after  which  a  sharp  decline  in  value  occurs.  A  student  is 
motivated  by  being  assigned  to  his/her  top  three  choices  and  de-motivated  by  any  other  choice. 


Figure  13  -  Student  Preference  Value  Function 
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Skills  Breakdown 


From  the  follow-up  survey,  a  breakdown  of  each  broad  skill  area  was  calculated.  For 


General  and  Other  Skills  sets,  the  data  was  averaged  and  a  95%  confidence  interval  (half-width 


shown  in  Table  2  and  Table  3)  was  calculated.  Table  2  and  Table  3  show  the  data  results. 


Figure  14  and  Figure  15  graphically  depict  the  percentage  breakdowns  for  the  two  broad  areas. 


Table  2  -  General  Skills  breakdown  data  results 


General 

Skills 

SA 

Multi- 

Task 

Sensor 

Ops 

Discipline 

Risk 

Mgmt 

A/C 

Handling 

Crosscheck 

Basic 

Form 

Sys  GK 

Avg: 

20.26% 

21.68% 

10.15% 

10.40% 

8.20% 

9.99% 

9.19% 

3.65% 

6.49% 

SD: 

4.73% 

12.05% 

9.16% 

6.56% 

6.88% 

8.44% 

2.94% 

2.74% 

1.49% 

Cl: 

0.02% 

0.04% 

0.04% 

0.05% 

0.07% 

0.11% 

0.02% 

0.06% 

0.02% 

Table  3  -  Other  Skills  breakdown  data  results 


Other  Skills 

Prec  Fly 

LOS 

Interp 

Compart 

Attitude 

MSN  Prep 

Avg: 

1 .67% 

1.15% 

0.42% 

1 .28% 

0.85% 

SD: 

0.97% 

1 .92% 

0.52% 

Cl: 

0.02% 

0.04% 

0.01% 

General  Skills  breakdown 


Basic  Form 
4.0% 


Sys  GK 
6.2% 


8.2%  Discipline  10.4% 


10.6% 


Figure  14  -  General  Skills  Breakdown 
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Other  Skills  breakdown 


MSN  Prep 
15.3% 


Attitude 

22.4% 


Prec  Fly 
34.5% 


Compart 

7.5% 


LOS  Interp 
20.4% 


Figure  15  -  Other  Skills  Breakdown 

The  FCSS  did  not  query  opinions  concerning  “Attitude”  and  “Mission  Prep”.  To  prevent 
incorrect  weightings  from  a  misperception  and  misunderstanding  in  the  Other  Skills  set,  only  the 
data  from  the  follow-up  Updated  Skills  survey  was  used.  For  this  last  survey,  the  categories 
were  clearly  defined  and  potential  errors  associated  with  a  smaller  data  set  do  not  outweigh  the 
errors  associated  with  missing  data  from  the  FCSS. 

For  the  Specific  Skills  set,  the  data  from  each  aircraft  type  was  used  to  develop  the  matrix 
found  in  Table  4.  To  augment  the  aircraft  communities  with  small  sample  return,  representatives 


where  polled  to  provide  additional  input  on  the  various  weightings. 


Table  4  -  Specific  Skills  breakdown  by  aircraft 


A-10 

F-15C 

F-15E 

F-16 

F-22 

T-37/T-6 

T-38 

Bomber 

Communication/Listening 

12.5% 

20.3% 

16.9% 

20.2% 

8.2% 

20.0% 

17.0% 

16.0% 

Cognitive/Spatial  Thinking 

8.3% 

15.0% 

15.4% 

14.8% 

23.4% 

10.0% 

12.0% 

8.0% 

Information/Data  Processing 

21.3% 

13.8% 

1 1 .8% 

12.1% 

38.3% 

10.0% 

17.0% 

13.0% 

Advanced  Formation 

1.7% 

15.3% 

8.1% 

10.8% 

0.0% 

5.0% 

16.0% 

17.0% 

Crew/Flight  Coordination 

7.9% 

7.0% 

18.4% 

8.2% 

2.4% 

20.0% 

8.0% 

17.0% 

Mental  Calculations 

8.8% 

7.6% 

2.9% 

7.9% 

8.2% 

10.0% 

6.0% 

4.0% 

Quick  Thinking 

10.0% 

10.7% 

5.1% 

9.2% 

18.5% 

5.0% 

12.0% 

4.0% 

Target  ID 

24.5% 

5.7% 

19.1% 

14.0% 

0.0% 

5.0% 

6.0% 

8.0% 

Map  Reading/Interpretation 

5.0% 

4.6% 

2.2% 

2.8% 

1.0% 

15.0% 

6.0% 

13.0% 

Since  the  focus  of  this  research  effort  was  on  fighter  aircraft  assignments  from  ENJJPT, 


the  FCSS  was  geared  toward  fighter  aircraft  communities.  Because  of  this  focus,  limited 
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responses  were  obtained  from  the  trainer  and  bomber  communities.  To  properly  convey  the 
trainer  and  bomber  aircraft  skill  needs,  further  results  need  to  be  obtained  from  these 
communities.  The  F-15C,  F-15E,  F-16  and  F-22  sample  sizes  were  sufficiently  large  to  provide 
statistically  accurate  data  with  one  exception.  As  stated  before,  the  F-16  Map  Reading  category 
was  changed  from  0%  to  4.8%  and  the  remaining  skill  areas  reduced  by  a  proportional  amount. 
Sub-Skills  score  development 

Calculating  the  actual  score  to  be  input  into  the  model  involves  accessing  a  vast  amount 
of  data  from  the  grades  input  by  the  instructors  over  the  course  of  pilot  training  for  each  student. 
Conducted  manually,  this  is  a  labor  intensive  effort.  Currently,  the  TIMS  system  provides  data 
output  based  on  tables  developed  to  support  the  MASS  and  is  automatically  extracted  whenever 
a  “run”  is  requested. 


Table  5  -  Sub-skills  tabulation 


Low 

High 

Factor 

Shift 

Communication/Listening 

-10 

25 

0.0286 

10 

Cognitive/Spatial  Thinking 

0 

30 

0.0333 

0 

Information/Data  Processing 

0 

10 

0.1000 

0 

Advanced  Formation 

-15 

50 

0.0154 

15 

Crew/Flight  Coordination 

0 

15 

0.0667 

0 

Mental  Calculations 

0 

20 

0.0500 

0 

Quick  Thinking 

0 

20 

0.0500 

0 

Target  ID 

0 

20 

0.0500 

0 

Map  Reading/Interpretation 

0 

20 

0.0500 

0 

The  concept  behind  the  scoring  is  to  determine  the  deviation  from  Maneuver  Information 


File  (MIF)  for  each  grade,  average  the  deviation,  and  assign  a  point  value.  Using  DA  principles, 


the  data  must  first  be  conditioned.  Table  5  is  an  example  of  how  the  data  would  need  to  be 


conditioned  for  the  database  table.  For  each  skill  area  a  low  and  high  deviation  can  be 


calculated.  For  the  low  value,  the  lowest  possible  grades  while  still  achieving  progression  in 


accordance  with  the  current  syllabus  are  assumed.  For  the  high  value,  the  opposite  is  assumed. 


Both  extremes  are  highly  unlikely  but  do  represent  the  extremes.  Since  it  is  possible  to  achieve 
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negative  scores,  the  “shift”  value  establishes  “0”  as  the  standard  reference  point.  The  “factor” 
normalizes  the  raw  scores  so  each  skill  area  has  a  relative  score  from  0  to  1 . 


Sensitivity  Analysis 


As  part  of  the  data  analysis,  the  areas  of  the  Specific  Skills  set  were  compared  between 


aircraft  types.  A  table  was  developed  within  the  model  to  show  the  relative  values  between 


aircraft  types  for  a  given  data  set.  Figure  16  shows  an  example  of  this  relationship. 


Comm  / 

Cog  Thinh< 

Info/Data 

Crew/Fit 

Mental 

Quick 

Map  Reading/ 

Listening 

Thinf 

Processing 

Coord 

Calc 

Thinking 

Interpretation 

Raw  Score: 

10 

5  V 

10 

Jl 

3 

4 

5 

4 

10 

A-10 

F-15C 

F-16 

F-22 

T-6/T-37 

T-38 

Bomber 

Specific  A/C  Score: 

43.642 

42.088 

37.242 

39.485 

54.131 

40.538 

44.130 

42.530 

Top  3  Best  Fit:  f 

#1 

\  #2 

#3 

F-22 

JT-38 

A-10 

54.131 

/44.130 

43.642 

<u 

<0 

> 


Skill  Comparison/Makeup 


0.600 

0.500 

0.400 

0.300 

0.200 

0.100 

0.000 


.38 

= 

.21 

.14 

.12 

.12 

m 

.10 

.17 

.13 

.12 

.10 

.12 

.05 

,ii 

.10 

.09 

nS) 


& 

N° 


f{y  ($■ 

* 

Aircraft 


A 


& 


■  Map  Reading/ 
Interpretation 

□  Target  ID 

□  Quick 
Thinking 

□  Mental 
Calc 

■  Crew/Fit 
Coord 

□  Adv  Form 

□  Info/Data 
Processing 

ECog  Think/ 
Spatial  Think 

□  Comm/ 
Listening 


Figure  16  -  Skill  Comparison  Analysis  #1 

In  this  example,  Information/Data  Processing  can  be  seen  as  the  most  dominate  skill, 


followed  by  Communication/Listening.  To  further  understand  the  relationship,  the  particular 


skill  values  have  been  added.  By  incrementally  changing  Info/Data  Processing,  it  can  be  seen 
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that  once  the  value  reaches  5.4,  the  best  fit  aircraft  changes  to  a  Bomber  (Figure  17).  Each  of  the 
nine  skill  areas  can  be  assessed  in  this  manner. 


When  two  or  more  aircraft  have  an  identical  score,  the  default  will  be  the  furthest  aircraft 
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to  the  right.  However,  at  one  decimal  place  for  the  score  values,  there  are  1x10  possible 


combinations  meaning  the  likelihood  of  an  identical  match  is  extremely  low.  With  the  addition 


of  the  student  preference,  the  probability  of  a  match  is  even  lower.  However,  if  an  identical 


match  occurred,  the  decision  maker  would  still  have  the  ability  to  choose  the  assignment. 


Specific  Skills 

Comm/ 

Listening 

Cog  Tiling 
Spatir  l 
Thinll 

Info/Data  _  A,_ 

_  AcMForm 

Processing 

Crew/Fit 

Coord 

Mental 

Calc 

Quick 

Thinking 

Target  ID 

Map  Reading/ 
Interpretation 

Raw  Score: 

10 

5  \ 

5.4  Jl 

3 

4 

5 

4 

10 

A-10 

F-15C 

F-ISP-11^  F-16 

F-22 

T-6/T-37 

T-38 

Bomber 

Specific  A/C  Score: 
Top3  Besat  Fit: 


0.400 
0.350 
0.300 
«  0.250 
1  0.200 
>  0.150 
0.100 
0.050 
0.000 


33.844 

35.740 

31.814 

#1 

\  #2 

#3 

Bomber 

V-22 

T-38 

36.550 

^.513 

36.310 

33.919  36.513  35.938  36.310 


36.550 


Skill  Comparison/Makeup 


■ 

.21 

u 

.12 

.07 
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05 
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.07 

.12 

.10 

.12 

.11 

.10 

.09 

.U/ 

1 

1 

1 

1 

.05 

1 

1 

1 

^  Hp'  ^  <£ 
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■  Map  Reading/ 
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□  Target  ID 

□  Quick 
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■  Crew/Fit 
Coord 

□  Adv  Form 

□  Info/Data 
Processing 

□  Cog  Think/ 
Spatial  Think 

□  Comm/ 
Listening 


Figure  17  -  Skill  Comparison  Analysis  #2 


Relating  UPT  events  to  Skills 

A  key  objective  of  the  ENJJPT  MASS  Working  Group  was  to  develop  a  list  of 


relationships  between  events  taught  in  UPT  (both  current  and  possibly  future)  and  the  Skill  areas 
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and  sets  developed  from  the  FCSS  and  other  surveys.  Appendix  G  lists  each  of  the  Broad  Skill 
sets  with  the  accompanying  sub-skill  areas.  Using  a  brainstonning  session  during  the  ENJJPT 
MASS  Working  Group,  each  skill  area  was  tied  to  one  or  more  UPT  events  which  are  listed  in 
Appendix  G. 

In  addition,  the  group  provided  feedback  on  the  need  for  “narrow”  definitions.  After 
each  skill  and  UPT  event,  a  definition,  objective,  or  question  is  stated  to  help  define  the  event 
and  the  relationship  to  the  particular  skill.  The  current  syllabus  provides  Phase  Training 
Standards  (PTS)  and  general  definitions  but  because  the  syllabus  is  not  focused  toward  skill  sets, 
the  skill  areas  need  proper  explanations. 

Table  6  -  PTS  Excerpt  from  ENJJPT  Syllabus  (May  2004) 


b  Enroute 

a. 

Flight  mission  profile  requiring 

a.  Maintains  course  ±5°  or  a  valid 

(1)  TACAN  course  intercepts 

accomplishment  of  these  maneuvers. 

intercept. 

inbound,  outbound,  immediately 

*>■ 

FLIP  departure,  enroute,  and  arrival 

b.  Maintains  arc  ±2  NM  or  a  valid 

after  station  passage,  and  maintam 

publications  available  in  aircraft. 

intercept. 

course. 

c- 

Predetermined  or  directed  route  of 

c.  Obtain  required  data  and  calculate 

(2)  TACAN  arc-to-radial  intercepts, 

flight  to  be  flown. 

PIiiIIImJ  . . 1  TT'il'  111  Ml  III  Mil'll  III  jjjnl  III 

d. 

Predetermined  or  directed  target 

^mffight  planning.  •v 

fix. 

/ 

d.  Initiate  necessary  corrections  to 

f  (3)  Groundspeed  check  > 

ve 

AF  Form  70.  ( 

target  for  fix-to-fix.  and  arrive  within  3 

f  (4)  TACAN  fix-to-fix  navigation. 

y 

Navigation  computer.  ' 

yNM  radius.  > 

\.  (5)  Maintain  enroute  time  and  fuel/ 

g- 

Day  or  night. 

e!  Aisoon  as  practicable  afteu»«fM 

^Nt^a 

Special 

each  enroute  fax.  fail  Utilize  actual  time 

i. 

Night  navigation  techniques  are 

of  arrival  at  the  fix,  actual  fuel 

(7)  Perform  TACAN  holding. 

practiced 

remaining  at  the  fix.  and  estimated  tune 

2- 

Prescribed  heading  and  level-off 

of  arrival  for  next  fix  for  updating 

altitude. 

inflight  planning. 

3. 

Day  or  night. 

f.  Able  to  update  precomputed  fuel 

4. 

Holding  pattern  as  described  by 

and  time,  as  necessary. 

controller  or  FLIP  document. 

g.  Plan  and  execute  and  enroute 

5. 

Given  an  EFC,  EAC,  or  an  ETA. 

descent  which  considers  fuel  efficiency 
and  results  in  a  smooth  transition  of 

final  approach 

h.  Performs  entry  procedures  and 
maintains  designated  pattern  LAW 

AFM  11-217  and  Flight  Manual 
procedures. 

As  an  example,  Table  6  shows  an  excerpt  of  the  PTS  from  the  ENJJPT  Syllabus.  The 
first  column  is  the  “skill”  area,  the  second  the  conditions  under  which  the  student  will  perfonn 


the  maneuver,  and  the  third,  the  standard  the  student  must  achieve.  For  a  “fix-to-fix”  maneuver, 
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the  only  criterion  is  the  student  must  “initiate  necessary  corrections  to  target  for  fix-to-fix,  and 
arrive  within  3  NM  radius”  (ENJJPT  Syllabus,  May:68).  Under  the  skills-based  construct,  the 
instructor  presents  the  student  with  the  concept  of  using  the  60-to-l  rule,  the  rule  of  5’s,  etc.  to 
give  the  tools  and  skills  behind  the  fix-to-fix  maneuver.  The  grade  given  to  the  student  is  a 
reflection  of  his/her  ability  to  adapt  to  this  skill  and  properly  apply  the  skill  to  the  maneuver. 

The  definitions  as  shown  in  Appendix  G  are: 

Table  7  -  Skill  definition  excerpts 

Aircraft  Specific  Mental  Calculations  Fix-to-fix  Applies  proper  mental  calculations  to  achieve  a  successful  fix-to-fix 

Arcraft  Specific  Information/Data  Processing  Fix-to-fix  Incorporates  all  available  information  to  fly  a  proper  fix-to-fix 

Bottom  Line 

When  a  student  graduates  from  pilot  training,  they  need  to  have  a  defined  skill  set  the 
follow-on  training  units  can  use  and  further  develop.  The  data  collected  shows  there  is  a 
commonality  among  many  skills  but  nine  separate  skills  vary  among  the  aircraft  types.  The 
feedback  received  from  the  field  overwhelming  states  every  graduate  of  pilot  training  needs  to 
posses  a  solid,  robust  foundation  of  basic  skills.  These  skills  are  encompassed  in  the  General 
Skills  set  in  this  model.  For  the  individual  communities,  a  solid  framework  of  specialized  or 
Specific  Skills  needs  to  exist.  This  framework  is  where  the  IFF  and  the  RTUs  focus  their 
energies  further  developing  the  newly  minted  pilot.  Finally,  the  more  esoteric  or  higher  skills 
complete  the  pilot.  Although  desired,  these  skills  are  not  as  crucial  to  the  initial  development  of 
a  pilot. 
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V.  Model  Development  and  Analysis 


Overview 

This  chapter  presents  the  development  and  use  of  the  assignment  model.  The  intent 
behind  the  model  was  to  provide  a  simple,  user  friendly  application  to  assist  the  decision  maker 
in  determining  future  placement  of  a  UPT  class  of  students.  The  goal  was  to  provide  a  model 
utilizing  a  desktop  PC  with  standard  software  available  to  all  USAF  users.  For  this  application, 
Microsoft  Excel  was  used  as  the  base  spreadsheet  program. 

Model  Input 

Figure  18  shows  the  input  page  for  Student  Preference.  For  each  student,  the  eight 
choices  are  rank  ordered  and  put  into  the  spreadsheet.  The  model  contains  a  search  function  for 
an  aircraft  name  and  will  return  a  “0”  score  if  an  aircraft  is  not  listed  in  the  preference.  To 
prevent  a  student  from  having  an  undue  advantage  by  not  listing  an  “unwanted”  aircraft,  all  eight 
blocks  should  be  completed  unless  the  decision  maker  is  aware  a  particular  aircraft  will  not  be 
available  (eg  F-22  and  later,  the  F-15C). 


Note:  Although  preferences  are  not  case  senslty,  the  syntax  must  match 
X2  to  AE2  cells  (i.e.  "A-10"  can  not  be  entered  as  "alO") 


Student  ID 

Student 

Name 

1 

2 

3 

4 

5 

6 

7 

8 

1 

A 

F-15C 

F-15E 

F-16 

A-10 

F-22 

T-6/T-37 

T-38 

Bomber 

2 

B 

F-15E 

F-15E 

F-16 

F-15C 

F-22 

T-6/T-37 

T-38 

Bomber 

3 

4 

5 

C 

Bomber 

F-15C 

F-16 

F-15E 

A-10 

F-22 

T-6/T-37 

T-38 

Figure  18  -  Student  Preference  input  page 

Figure  19,  Figure  20,  and  Figure  2 1  show  the  input  pages  for  entering  the  student  skill 
score  data.  The  actual  values  for  each  skill  area  are  obtained  from  TIMS  and  the  combined  score 
input  in  the  model.  The  low  and  hi  values  are  the  range  of  possible  raw  scores. 
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General  Skills  Note:  If  a  cell  is  blank,  a  "0"  is  assumed  which  may  not  be  the  low  value 

Raw  Scores:  for  the  skill  area  and  will  result  in  a  skills  score. 


SA 

Multi- 

Tasking 

Sensor 

Ops 

Discipline 

Risk  Management 

Aircraft  Handling 

Crosscheck 

Basic 

Formation 

System 

Knowledge 

Low: 

-10 

0 

0 

0 

-15 

0 

0 

-10 

0 

Hi: 

25 

30 

10 

50 

15 

25 

50 

20 

20 

Stud  1 

-10 

0 

0 

0 

-15 

0 

0 

-10 

0 

Stud  2 

25 

30 

10 

50 

15 

25 

50 

20 

20 

Stud  3 

20 

25 

8 

40 

10 

20 

40 

15 

15 

Figure  19  -  General  Skills  input  page 


Other  Skills  Raw 

Scores: 

Attitude 

Msn  Plan/ 
Prep 

Prec  Fly 

LOS 

Interpret 

Compart 

Low: 

-10 

0 

0 

0 

-15 

Hi: 

25 

30 

10 

50 

15 

Stud  1 

-10 

0 

0 

0 

-15 

Stud  2 

25 

30 

10 

50 

15 

Stud  3 

20 

25 

8 

45 

10 

Figure  20  -  Other  Skills  input  page 


Specific  Skills  Raw 
Scores: 


Comm/ 

Listening 

Cog  Think/ 
Spatial  Think 

Info/Data 

Processing 

Adv  Form 

Crew/Fit 

Coord 

Mental 

Calc 

Quick 

Thinking 

Target  ID 

Map  Reading/ 
Interpretation 

Low: 

-10 

0 

0 

-15 

0 

0 

0 

0 

0 

Hi: 

25 

30 

10 

50 

15 

20 

20 

20 

20 

Stud  1 

-10 

0 

0 

0 

-15 

0 

0 

0 

0 

Stud  2 

25 

30 

10 

50 

15 

20 

20 

20 

20 

Stud  3 

20 

25 

8 

45 

10 

15 

15 

15 

15 

Figure  21  -  Aircraft  Specific  Skills  input  page 


UPT  Grading 

To  comprehend  how  the  skill  score  is  obtained,  the  grading  scheme  must  first  be 
understood.  Table  8  is  an  excerpt  from  the  AETC  pilot  training  syllabus  and  is  similar  for  all 
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levels  of  flying  training.  When  a  student  is  first  introduced  to  a  maneuver,  the  instructor 
normally  demonstrates  the  maneuver  prior  to  the  student  performing  and  practicing  the  event.  If 
the  student  does  not  perform  the  maneuver  on  the  same  sortie  it  was  demonstrated,  the  grade  will 
be  “NG”  or  1.  Also,  when  a  student  is  solo  and  the  instructor  can  not  observe  the  performance, 
the  grade  will  be  NG.  The  levels  of  performance  for  each  grade  level  are  shown  in  the  table. 

The  MIF  is  a  table  of  levels  of  expected  performance.  A  student  needs  to  achieve  the  established 
MIF  at  the  end  of  a  block  of  training.  A  category  of  training,  for  example  contact,  consists  of 
many  blocks  of  training  with  progressively  increasing  MIF  levels. 

Table  8  -  Grade  Definitions 

Proficiency  Maneuver  Grades  MIF  Level  Description 

No  Grade  (NG)  1  Enter  NG  on  the  record  of  training  when  the  maneuver  is  demonstrated  by  an  instructor 
pilot  on  a  dual  sortie.  On  solo  sorties  enter  NG  for  maneuvers  flown,  but  not  observed. 

Unsatisfactory  (U)  2  The  student  is  unsafe  or  unable  due  to  lack  of  sufficient  knowledge,  skill  or  ability  to 
perform  the  operation,  maneuver,  or  task. 

Fair  (F)  3  The  student  performs  the  operation,  maneuver,  or  task  safely  but  has  limited  proficiency. 
Deviations  occur  that  detract  from  performance  and/or  verbal  prompting  was  required  from  the  instructor. 
Good (G)  4  The  student  performs  the  operation,  maneuver,  or  task  satisfactorily.  Deviations  occur  that  are 
recognized  and  corrected  in  a  timely  manner  without  verbal  prompting  from  the  instructor. 

Excellent  (E)  5  The  student  performs  the  operation,  maneuver,  or  task  correctly,  efficiently,  and  skillfully. 
Minor  deviations  occur  that  do  not  detract  from  the  overall  performance. 


Model  Calculations 

There  are  several  calculations  performed  by  the  model  to  provide  a  best  fit  result.  The 
two  main  results  are  the  Pilot  Ability  score  and  the  Best  Fit  match.  The  Pilot  Ability  score  is 
given  by: 


Pilot  Ability  (PA)  Score  =  General  Skills  Score  + 

Other  Skills  Score  +  Average  (Aircraft  Specific  Skills  Scores)  (2) 


The  PA  score  is  used  for  overall  ranking  and  comparison  of  the  students  in  a  particular  class. 
This  score  will  be  used  for  award  determinations,  Distinguished  Graduate  certificates,  etc. 
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The  Best  Fit  determination  only  considers  the  Aircraft  Specific  Skills  and  the  student’s 
preference.  A  score  for  each  aircraft  type  is  obtained  and  the  highest  three  scores  produce  an 
aircraft  match.  Each  aircraft  score  is  given  by: 

Aircraft  Score  =  Specific  Skills  Score  +  10%  of  Preference  Score  (3) 

The  skills  set  score  begins  with  the  grades  received  by  the  student  on  every  sortie  for 
every  maneuver/skill  event.  Figure  22  shows  the  general  flow  used  in  the  calculations.  The 
number  in  parenthesis  indicates  the  applicable  equation  number  or  figure. 


Eq  (4)  Eq  (5),  (6),  and  (7) 


Figure  22  -  General  Calculation  Flow 


Specific  Model  Calculations 

The  sub-skill  score  is  the  overall  average  grade  deviation  from  the  expected  performance 
on  a  given  sortie.  The  sortie  classification  weighting  is  applied  for  each  grade  received 


#_  event  _  types 

Sub-Skill  Score  =  ^ 


(Grade s  -  MIFs  )*  SC W 


5=1 


(4) 


Where  Grades  is  the  numeric  grade  received  during  the  sortie,  MIFS  is  the  standard  for  the  sortie, 
n  is  the  number  of  sortie  where  the  maneuver/item  was  flown  and  graded  and  SCW  is  the  Sortie 
Class  Weight  (see  Figure  9). 
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The  PA  score  is  calculated  with  all  three  skill  sets  using  the  following  equations: 

f  9  A 


General  Skills  Score  = 


^S,+Shift,)*F, 


V  1=1 


=  100 


J 


f  5 


Other  Skills  Score 


V  i=i 

f  9 


100 


Aircraft  Skill  Score  = 


Y,(SI  + Shift,)*  F, 
=  1 


=  100 


(5) 

(6) 

(7) 


V  i=i  J 

Where:  St  =  Raw  Score,  Shift,  =  Shift  for  skill  i,  F,  =  Factor  for  skill  i,  and  Wj  =  Aircraft 
weighting  for  skill  i. 

Since  each  aircraft  type  has  a  different  weighting  for  the  sub-skills  in  the  Specific  Skills 
set,  an  aircraft  factor  was  applied  to  the  overall  skill  set  score.  When  detennining  the  PA  score, 
the  eight  aircraft  specific  scores  are  averaged.  Each  of  the  above  equations  is  multiplied  by  100 
for  ease  of  reading  the  scores  when  entering  the  values  into  the  Excel  model. 

As  stated  previously,  a  shift  and  factor  are  used  to  normalize  the  data  between  sub-skill 
sets  prior  to  the  weighting  being  applied  and  are  given  by: 

Shift  for  skill  i  =  -  low  value  (8) 

Fj  =  Factor  for  skill  i  = - - -  (9) 

Hi  -Low 

Figure  23  shows  an  example  of  the  sub-skill  determination  demonstrating  how  the  shift 
and  factors  are  calculated  and  used. 
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Low  =  -10 
High  =  10 

Fopfot*  ■ —  —  OS 

Shift  =  10 

E  dvlUl  •  «_/ 

10  — (—10) 

Score 

Value 

-10 

0 

0 

.5 

5 

.75 

10 

1 

Figure  23  -  Example  Sub-Skill  calculation 

For  the  model,  a  starting  place  for  the  skill  set  scores  range  from  0  to  100.  All  of  the 
weighted  scores  result  in  a  value  from  0  to  1 .  Note  the  high  and  low  values  and  the  score  ranges 
are  arbitrary.  As  the  actual  database  tables  are  developed,  these  values  may  change  to  reflect  the 
actual  scores  obtained  by  the  students  in  pilot  training. 

Sub-Skill  Calculation  Example 

The  following  is  an  example  of  how  the  sub-skill  calculation  is  detennined.  There  are 
close  to  10,000  recorded  events  in  UPT  for  each  student.  This  example  is  a  much  simplified 
version  but  will  show  the  general  concept. 

For  each  stage  of  training,  a  particular  event  may  have  a  MIF  that  changes  as  the  student 
gains  experience.  These  increasing  expectations  are  captured  in  a  master  table  database.  Again, 
for  simplification,  a  set  MIF  is  used.  From  Equation  (4),  the  sub-skill  score  is  determined  using 
average  of  the  deviations  from  MIF  then  summed  over  each  event  type.  For  example,  the  skill 
Map  Reading  is  comprised  of  the  following  UPT  events:  Pilotage,  Heading  Control,  Mission 
Planning  (Low  Level),  and  Temporary  Flight  Restrictions  (TFR’s).  Note  that  TFR  awareness  is 
not  currently  a  graded  item  in  UPT  and  is  one  of  many  suggested  changes  to  the  syllabus. 
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Table  9  -  i 

Example  IV 

[IF  Calculation 

Event 

MIF 

Average 

deviation 

from  MIF 

Min 

Max 

#  times  event 

practice/performed 

Average 

Pilotage 

+3 

5 

-15 

30 

15 

.33 

Heading  Control 

+4 

10 

-99 

50 

50 

.2 

Mission  Planning 

(Low  Level) 

+3 

12 

-10 

20 

10 

1.2 

TFR’s 

+4 

10 

-19 

10 

10 

1 

The  low  and  high  values  are  calculated  based  on  the  UPT  rules  set.  A  “No  Grade”  has  a 
grade  value  of  1  and  can  only  be  demonstrated  once  by  an  instructor  or  any  time  solo.  To  pass  a 
block  of  training,  a  student  must  achieve  a  grade  equal  to  or  greater  than  MIF  on  the  last  ride  in  a 
block  of  training.  The  minimum  score  is  calculated  accounting  for  the  least  number  of  points 
available  where  the  student  is  furthest  away  from  the  MIF  in  the  negative  direction  but  still 
achieving  MIF  by  the  end  of  the  block  of  training.  The  maximum  is  detennined  from  a  student 
achieving  “excellent”  grades  on  each  performance  of  the  maneuver  and  the  deviation  from  MIF 
summed.  The  student  in  this  example  (Table  9)  would  receive  a  score  for  Map  Reading  of  2.73. 
Note,  because  Heading  Control  is  a  skill  not  just  perfonned  and  graded  in  low  level,  the  number 
of  completed  events  is  much  greater  than  the  other  areas. 

Model  Output 

Figure  24  shows  the  model  output.  The  entire  class  is  listed  with  an  overall  ability  score, 
which  is  comprised  of  the  three  broad  skill  scores,  and  the  top  three  aircraft  in  rank  order.  As 
part  of  the  calculations,  a  “cut-off’  point  is  used.  If  the  student’s  ability  score  is  less  than  the 
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cut-off,  the  term  “washout”  is  displayed  indicating  the  student  has  failed  to  meet  a  minimum 
level  of  ability.  This  level  can  be  adjusted  from  the  initially  set  score  of  50.  Normally,  the 
regular  syllabus  contains  provisions  for  students  who  do  not  meet  minimum  standards.  If  the 
model  is  used  for  mid-term  feedback,  the  results  page  can  provide  a  potentially  valuable 
indicator  that  a  student  is  not  meeting  the  minimums  even  if  they  have  not  met  the  “triggers”  in 
the  syllabus. 

Student  ID  Student  Name  Pilot  Ability  Score  _ 


Best  Fit 

Second  Best  Fit 

Third  Best  Fit 

1 

0.000 

washout 

washout 

washout 

2 

261.250 

F-15E 

F-16 

F-15C 

3 

217.353 

Bomber 

F-15C 

F-16 

4 

33.024 

washout 

washout 

washout 

5 

33.024 

washout 

washout 

washout 

Figure  24  -  Results/Output  Table 

The  decision  maker  can  request  a  “run”  at  any  time  during  the  course  of  pilot  training. 
The  run  output  will  provide  the  necessary  data  for  input  into  the  model.  Follow-on  development 
of  the  model  will  automate  this  process. 

Model  Uses 

The  primary  use  of  the  model  is  obviously  for  student  assignment  recommendations. 
Once  the  infonnation  is  entered  into  the  spreadsheet  model,  the  top  three  aircraft  results  are 
displayed.  These  three  aircraft  best  match  a  student’s  skills  and  current  ability.  In  theory,  the 
student  will  perform  the  best  by  receiving  the  suggested  aircraft  assignment.  By  assigning  a 
student  based  on  his/her  strengths,  remedial  training  in  later,  more  expensive  stages  of  training 
should  be  reduced  or  eliminated.  As  with  any  model,  the  result  is  only  a  recommendation  and 
the  ultimate  choice  still  lies  with  the  decision  maker. 

The  model  also  has  the  side  benefit  of  providing  feedback  at  any  time  during  the  course 
of  pilot  training.  By  conducting  a  run,  the  instructor  core  can  view  the  current  student  rankings 
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within  a  class  and  the  skill  levels  of  each  student.  If  a  particular  student  is  weak  in  a  skill  area, 
the  instructors  can  use  this  information  to  focus  their  efforts  and  assist  the  student  in  improving 
this  weak  skill  area. 

Once  students  receive  their  aircraft  assignments,  the  model  can  be  again  used  to  identify 
remaining  deficiencies  as  related  to  their  assigned  aircraft.  One  possibility  is  to  use  a  “flexible” 
syllabus.  Toward  the  end  of  the  UPT  syllabus,  two  to  three  “free  profile”  sorties  could  exist  for 
the  instructors  to  use  for  additional  training  for  the  student.  These  sorties  would  be  dedicated 
toward  improving  the  student’s  weakest  skill  areas  prior  to  graduation.  For  example,  a  student  is 
assigned  to  an  F- 16  but  still  has  a  weak  area  of  Target  Identification.  The  additional  sorties  can 
be  dedicated  to  target  identification  practice. 

Overall 

As  previously  stated,  no  two  students  leam  at  the  same  rate.  For  some,  the  “light  bulb” 
doesn’t  come  on  until  well  into  pilot  training.  It  is  said  that  given  enough  hours  and  sorties, 
anyone  can  fly.  The  reality  is  there  is  not  an  infinite  supply  of  training  funds  and  there  is  a  limit 
on  time  and  space  for  training  students  in  UPT.  The  syllabus  guides  the  instructors  in  the 
development  of  a  new  pilot  providing  a  framework  and  timeline.  At  some  point  in  the  training 
timeline,  a  decision  must  be  made  to  the  student’s  future.  In  many  cases,  this  assignment 
decision  will  impact  the  student  for  their  20+  year  career. 

The  model  presented  here  is  a  tool  to  aid  the  decision  maker  in  assigning  students  based 
on  their  current  skill  and  ability.  No  model  is  perfect  but  the  information  provided  can  assist  in 
making  the  best  choice  fit  for  the  student  at  the  designated  time  in  the  program. 
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VI.  Recommendations 


Overview 

The  overarching  goal  of  this  research  effort  was  to  develop  an  assignment  model  for  use 
at  ENJJPT  and  ultimately,  as  part  of  the  USAF  UPT  student  assignment  process.  A  model, 
though,  is  only  useful  if  the  end-user  is  able  to  implement  the  necessary  provisions.  This  chapter 
discusses  the  following  general  list  of  recommended  changes  for  implementation: 

1)  Change  USAF  and  AETC  training  policy  for  all  pilot  training  to  be  skills  based 

2)  Incorporate  the  assignment  model  and  skills  concept  into  the  ENJJPT  ’08  plan 

3)  Use  the  model  to  guide  decision  makers  on  aircraft  assignment  of  UPT  students 

4)  Modify  all  UPT  training  syllabi  to  reflect  the  skills  based  approach 

5)  Develop  further  models  for  use  as  feedback  tools  at  follow-on  training  units 

6)  Continue  to  define  and  refine  skill  sets  for  every  level  of  training 

7)  Develop  procedures  for  use  as  a  mid-course  feedback  and  guidance 

8)  Modify  the  UPT  syllabi  using  feedback  to  improve  skills  of  weaker  students 
Policy  Change 

The  single  largest  paradigm  shift  is  the  concept  of  skills  based  training,  evaluation  and 
assignment  in  the  UPT  environment.  AETCI  36-2205  is  the  governing  instruction  on  training 
conduct  within  AETC.  To  reflect  the  change  in  philosophy,  this  regulation  would  need  to  be 
rewritten  to  present  the  concept  of  skills  based  training,  present  the  assigmnent  model,  and  direct 
implementation  at  the  wing  level. 

Any  other  training  policy  directives  from  AETC  must  also  be  addressed.  Since  the 
conduct  of  UPT  has  changed  relatively  little  over  many  decades,  leadership  support  is  key  to  the 
success  of  implementation  and  requires  thorough  understanding  of  why  this  program  should  be 
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executed.  The  skills  based  method  provides  a  cradle  to  grave  approach  to  pilot  production.  Skill 
sets  at  each  stage  of  upgrade  will  be  defined  resulting  in  a  focused  training  effort  with  a  goal  of 
providing  the  CAF  with  a  pilot  possessing  robust,  defined  and  necessary  skills  to  effectively 
complete  today’s  complex  combat  mission. 

ENJJPT  ’08  Plan 

The  ENJJPT  ’08  plan  is  still  in  development  and  timing  is  crucial  for  inclusion  into  the 
final  concept.  As  part  of  the  plan,  the  syllabus  for  each  phase  of  training  at  Sheppard  is  being 
rewritten.  By  adopting  the  skills  based  approach,  these  new  syllabi  can  be  written  from  the  skills 
perspective  with  a  fresh  look  at  each  maneuver  flown.  The  missions  can  be  developed  with  a 
clear  objective  stated  for  the  sortie  as  to  what  skill  is  introduced,  practiced,  and  evaluated. 

Since  ENJJPT  is  governed  by  the  SC,  buy-in  from  the  NATO  partners  is  necessary 
throughout  the  development  of  the  new  training  program.  The  skills  concept  needs  to  be 
introduced  at  the  earliest  stage  of  the  plan  development  to  provide  the  greatest  opportunity  for 
understanding  and  acceptance  by  the  SC. 

Assignment  Model 

The  core  of  this  effort  was  the  development  of  the  assignment  model.  This  model  can  be 
used  by  the  decision  maker  as  a  tool  to  objectively  determine  the  best  aircraft  assignment  for  the 
student.  As  stated  before,  the  model  is  but  one  of  many  tools  to  be  utilized.  Use  of  the  model 
will  move  the  current  process  away  from  rank-ordered  selection.  Top  perfonning  students  most 
likely  will  not  see  any  benefit  from  the  model  but  for  the  weaker,  struggling  students,  the  model 
provides  a  means  of  fairness  and  assurance  concerning  their  assignment.  These  students  can 
know  they  are  being  assigned  to  the  best  assignment  for  their  individual  talents. 
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Changes  to  UPT  and  the  Syllabus 

The  current  training  approach  in  UPT  is  to  fly  a  set  of  maneuvers  to  achieve  an  objective. 
The  orderly  flow  of  sorties  and  specific  set  of  maneuvers  constrains  the  instructor  to  a  somewhat 
scripted  approach  often  preventing  the  practice  of  needed  skills.  The  proposed  approach  to 
training  would  allow  an  instructor  greater  freedom  to  focus  on  a  student’s  weak  areas.  A  training 
plan  can  be  developed  and  modified  as  the  student  improves  and  achieves  the  desired  levels  of 
ability. 

Appendix  G  shows  the  list  of  skills  and  related  UPT  events.  The  highlighted  events  are 
those  not  explicitly  graded  in  the  current  UPT  environment.  Some  of  these  areas  fall  into  several 
categories  of  maneuvers  and  some  are  not  currently  taught.  As  part  of  the  syllabus  examination 
and  refinement  of  the  skill  definitions,  a  thorough  review  of  events  and  maneuvers  should  be 
completed.  Where  necessary,  new  maneuver  definitions  should  be  inserted  and  others  combined 
or  eliminated. 

Codifying  the  events  as  they  relate  to  skills  and  providing  narrow  definitions  gives  the 
instructors  a  mission  objective  and  focus  for  the  sortie.  The  student  can  in  turn  understand  the 
skill  area  and  focus  their  preparation  efforts  to  the  understanding  and  application  of  the  skill.  For 
example,  a  student  needs  to  fly  aerobatic  maneuvers  (loop,  roll,  etc).  The  intention  is  not  to  fly 
precise  maneuvers  in  the  sky  or  for  an  air  show  but  rather  to  understand  the  concepts  of  aircraft 
control.  Clearly  stating  the  skill  area  in  the  syllabus  provides  the  student  a  definitive  focus. 

Specific  recommendations  resulted  from  the  FCSS  and  ENJJPT  MASS  Working  Group: 

1)  Students  should  brief  the  mission/sortie.  The  student  not  only  learns  and  practices 
communication  skills,  but  is  forced  to  craft  and  present  a  coherent  plan.  Additionally, 
they  are  developing  their  cognitive  thinking  skills. 
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2)  Students  should  continue  to  lead  in  fonnation.  There  should  be  less  emphasis  on  flight 
leadership  but  the  student  still  needs  to  have  basic  skills  in  this  area.  The 
recommendation  is  for  the  student  to  continue  leading  to  and  from  the  area  with  the 
instructor  taking  charge  in  a  greater  capacity  within  the  area  during  maneuvering.  The 
student  is  developing  cognitive  and  decision  making  skills  but  the  expectations  should 
be  lowered  concerning  flight  leadership. 

3)  The  instrument  training  should  be  shifted  down  to  the  T-6  and  a  greater  emphasis 
placed  on  aircraft  handling  and  formation  in  the  T-38. 

Follow-on  Training  Models 

A  key  benefit  from  this  type  of  model  is  the  use  of  the  mid-course  feedback  to 
probabilistically  predict  the  future  performance  of  the  student.  Through  evaluation  of  current 
abilities  and  application  of  required/desired  skills,  an  assessment  can  be  made.  Since  some 
students  comprehend  subject  material  at  varying  rates,  the  model  will  not  perfectly  predict  the 
future.  However,  for  the  majority  of  students,  the  model  will  closely  identify  their  potential. 

Having  predictive  models  at  various  stages  of  training  can  assist  the  instructors  in 
objectively  identifying  weak  areas  allowing  for  a  focused  training  plan  to  be  created  and  used. 
Conducting  remedial  training  at  less  expensive  stages  of  training  will  benefit  the  Air  Force  and 
most  likely  benefit  the  student.  Correcting  weak  areas  at  early  stages  of  training  would  allow  the 
student  to  continue  in  the  training  pipeline  without  having  to  deviate  to  another  MWS  due  to 
failure  from  these  deficiencies. 


52 


VII.  Future  Research 


Further  Validation  and  Interface 

There  are  two  primary  tasks  remaining  prior  to  model  implementation  -  completing  the 
validation  and  developing  a  technical  interface.  As  stated  multiple  times,  approval  to  conduct 
research  on  currently  enrolled  students  was  not  obtained  in  a  timely  manner.  As  a  result,  the 
model  could  not  be  completely  validated  with  actual  student  data.  Subject  Matter  Experts  were 
used  in  the  testing  and  initial  confirmation  of  the  model  but  further  validation  is  warranted.  The 
survey  in  Appendix  H  is  a  template  which  can  be  used  to  relate  student  skills  to  the  model.  The 
data  results  from  these  surveys  can  be  compared  to  the  AETC  database  and  model  refinements 
made  from  the  results. 

A  technical  interface  to  extract  grade  data  from  TIMS  needs  to  be  developed.  When  the 
decision  maker  requests  a  run,  the  transfer  of  data  needs  to  be  seamless  and  automatic.  The 
grade  data  is  crucial  to  the  model  operation.  Without  the  interface,  the  assignment  model  is  only 
a  concept. 

UPT  Changes 

One  of  the  recommendations  from  the  previous  chapter  is  to  consolidate  some  of  the 
current  UPT  grades.  Although  the  skills  listed  in  Appendix  G  have  associated  UPT  events,  a 
complete  review  of  maneuvers  taught  during  UPT  needs  to  be  accomplished  to  ensure  all  the 
skills  are  properly  matched  and  defined. 

Skill  grades  should  be  considered  over  a  student  receiving  individual  maneuver  grades. 
For  example,  a  student  flying  aerobatics  will  be  graded  in  loop,  cloverleaf,  barrel  roll,  etc.  A 
proposal  is  for  the  student  to  be  graded  in  the  skill  areas  represented  by  the  maneuvers  -  aircraft 
control,  precision  handling,  entry  parameters,  etc.  A  student  would  still  be  required  to  fly  the 
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variety  of  aerobatic  maneuvers  but  would  be  graded  not  on  the  maneuver,  but  on  their  ability  in 
the  particular  skill  area. 

Final  Thoughts 

Assignment  night  in  pilot  training  is  one  of  the  most  memorable  events  in  a  young 
aviator’s  career.  The  events  leading  up  to  their  selection  are  trying  and  at  times,  heartbreaking. 
UPT  is  only  the  start  for  the  student.  Follow-on  training  increases  in  intensity  and  the  new  pilot 
will  be  tested  to  the  maximum  of  their  capabilities  physically,  mentally,  and  emotionally. 
Providing  the  tools  necessary  to  achieve  success  is  the  primary  goal  of  UPT. 

Identifying  the  necessary  skill  sets  and  accomplishing  training  based  on  these 
requirements  is  the  best  approach  to  successfully  achieving  the  goal  of  equipping  pilots  for  their 
future  training  needs.  Doing  so  smartly  benefits  not  only  the  individual,  but  the  Air  Force. 
Reducing  the  need  for  remedial  training  and  retraining  saves  valuable  resources  and  funds.  The 
presented  model  can  play  a  significant  role  in  the  effort  to  properly  identify  and  assign  students 
to  maximize  their  talents  and  abilities. 
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Appendix  A  -  Fighter  Skills  Survey  #2 

Appendix  A  is  a  copy  of  the  Fighter  Core  Skills  Survey  (FCSS).  The  survey  was  sent  to 
all  fighter  units  in  the  CONUS.  Each  page  represents  a  new  screen.  The  survey  was  web  based 
and  data  collection  was  anonymous.  A  summary  of  the  data  results  can  be  found  in  Chapter  IV. 


1 H  Improving  the  Assignment  Selection  and  Pipeline  Production  of  UPT  Students  -  Microsoft  Internet  Explorer 

0®|X|| 

File  Edit  View  Favorites  Tools  Help 

* 

muirro 

Improving  the  Assignment  Selection  and  Pipeline  Production  of  UPT  Students 


Survey  Control  Number:  USAF  SCN  05-132  Fighter  Pilot 
Core  Skills 


Privacy  Notice 


The  following  information  is  provided  as  required  by  the  Privacy  Act  of  1974: 

Purpose:  The  purpose  of  this  survey  is  to  collect  data  and  views  on  skills  required  by  fighter  pilots.  The  information  will  be  used  to  better  evaluate  and  assign  students 
graduating  pilot  training.  This  research  is  being  conducted  by  AFIT  and  sponsored  by  19  AF/DOK. 

Routine  Use:  No  analysis  of  individual  responses  will  be  conducted  and  only  members  of  the  Air  Force  Institute  of  Technology  research  team  will  be  permitted  access 
to  the  raw  data. 

Anonymity:  We  would  greatly  appreciate  your  participation.  ALL  ANSWERS  ARE  STRICTLY  ANONYMOUS.  Thus,  you  should  not  include  your  name  anywhere  on  this 
questionnaire  unless  you  would  like  to  be  contacted  concerning  this  research. 

Participation:  Participation  is  VOLUNTARY.  No  adverse  action  will  be  taken  against  any  member  who  does  not  participate  in  this  survey  or  who  does  not  complete  any 
part  of  the  survey. 

•  Goal  of  this  research  is  to: 

o  Identify  core  skills  for  each  aircraft  and  mission  type 
o  Better  identify  students  priorto  assignment 

o  Identify  areas  to  tailor  student  skills  prior  to  follow-on  training,  based  on  aircraft  assignment 

•  All  information  and  responses  contained  in  this  survey  is  UNCLASSIFIED/FOR  OFFICIAL  USE  ONLY 

•  Responses  are  anonymous 

•  UPT  refers  to  UPT,  SUPT.  and  ENJJPT 


Instructions 


•  Base  your  answers  on  your  own  thoughts  &  experiences 

•  Please  make  your  answers  clear  and  concise  when  asked  to  answer  in  a  response  or  when  providing  comments 

•  Be  sure  to  select  the  correct  option  button  when  asked  because  when  you  move  on  you  cannot  come  back 


Contact  information:  If  you  have  any  questions  or  comments  about  the  survey  contact  Maj  Michael  Messer  at  the  number,  fax,  mailing  address,  ore-mail  address 
provided  below. 

Maj  Michael  “Strike”  Messer 
AFIT/ENS 

BLDG  641  2950  P  Street 
Wright-Patterson  AFB  OH  45433-7765 
Email:  michael.messer@afitedu 

Phone:  DSN  255-6565,  ext  4326,  commercial  (937)  255-6565,  ext  4326 


Screen  1 
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Section  I:  Individual  Background/Aircraft 


1  What  year  did  you  enter  the  Air  Force  (active  duty)?  1 1985  v  ] 

2  What  best  describes  your  highest  qualification?  |  SEFE  v  | 

3  What  is  your  current/primary  aircraft?  |A-10  v~i 

NOTE:  If  you  are  currently  an  instructor  in  a  T-series  aJc,  list  your  primary  MWS  above  and  select  "other  aircraft"  below  to  list  your  T-series 
a/c.  If  currently  not  flying,  list  your  most  recent  MWS. 


4  Have  you  ever  flown  any  other  aircraft  as  a  crew  member? 


If  yes  please  list  other  aircraft 


Screen  2 


Section  II:  Individual  Background/Hours 


5  Approxamate  number  of  hours  in  the  F-15E 


6  Total  number  of  military  flight  hours  (approx) 


7  Approximate  number  of  hours  instructing  in  F-15E 


8  In  what  combat  operations  have  you  participated?  (select  all  that  apply) 


□  DS/DS  (Gulf  War  1) 

□  ONW 

□  osw 

□  Kosovo 

□  ODF 

□  OJG 

□  OF 

□  OEF 

□  Noble  Eagle 

□  Other 

Screen  3 


Section  III:  Mission  Breakdown 


Based  on  your  primary  aircraft,  what  do  you  feel 

9  □  Air  Superiority 

10  □  Escort 

11  Q  Interdiction 

12  □  Ground  Attack 

13  |-|  CAS 

14  |— j  Time  Sensitive  Targeting 


the  primary  roles/missions  (select  all  that  apply) 

15  □  DEAD 

16  □  SEAD 

17  □  Killer  Scout 

18  □  FAC 

19  j-|  Other 

20  □  Other 


Screen  4 
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Section  III:  Mission  Breakdown  (continued) 

1 .  Advanced  Formation  Skills  (tactical,  FM,  ext  trail) 

13.  Map  Reading/Interpretation 

2.  Aircraft  handling 

14.  Mental  Calculations 

3.  Basic  Formation  Skills  (fingertip,  route,  close  trl) 

15.  Multi-Tasking 

4.  Cognitive/spatial  Thinking 

1 6.  Precision  Flying 

5.  Communication 

17.  Quick  Thinking 

6.  Compartmentalization 

18.  Risk  management 

7.  Crew/flight  Coordination 

1 9 .  Sensor  Operations 

8.  Crosscheck 

20.  Situational  Awareness  (SA) 

9.  Discipline 

2 1 .  System  Know! edge 

10.  Info/Data  Processing 

22.  Target  Identification 

11.  Line-of-Sight  Interpretation 

23.  Task  Prioritization 

12.  Listening 

Using  the  list  above  and  by  entering  its  corresponding  number  in  the  boxes  below,  rank  order  up  to  the  top  four  (Rank  1  being  the  highest), 

skills  vou  feel  are  reouired  to  flv  the  mission  in  F-1 5E 

Air  Superiority  MISSION 

Rankl 

^  Rank  2 

Rank  3 

|  Rank  4 

List  any  additional  skills  you  feel  are  required  to  fly  the  Air  Superiority  mission  seperated  by  comma's 

1 - 1 

Interdiction  MISSION 
[  Rankl 
|  Rank  2 
|  Rank  3 
|  Rank  4 

List  any  additional  skills  you  feel  are  required  to  fly  the  Interdiction  mission  seperated  by  comma's 


Screen  5 


Section  IV:  Board  Area  Skills 


Board  area  skills  needed  to  fly  your  aircraft. 


1  Basic  Skills  are  those  needed  by  a  pilot  to  fly  any  aircraft  -  fighter  or  otherwise.  Sub-areas  include  general  knowledge,  task 
management,  basic  aircraft  handling,  risk  management,  communication,  and  airmanship.  On  a  scale  of  1  to  10  (10  being  most 
important),  how  important  are  basic  skills  to  flying  your  aircraft? 

oooooooooo 

2  Instrument  Skills  are  those  needed  by  a  pilot  to  fly  an  aircraft  under  Instrument  Flight  Rules  in  both  VMC  and  IMC.  Areas  include 
procedures,  approaches  (both  normal  and  emergency)  and  precision  aircraft  handling.  On  a  scale  of  1  to  10  (10  being  most  important), 
how  important  are  basic  skills  to  flying  your  aircraft? 

oooooooooo 


3 


Mission  Skills  are  those  needed  by  a  pilot  to  fly  and  execute  a  tactical/combat  mission.  Sub-areas  include  situational  awareness, 
formation  skills,  advanced  aircraft  handling,  mission  preparation,  and  mission  execution.  On  a  scale  of  1  to  10  (10  being  most 
important),  how  important  are  basic  skills  to  flying  your  aircraft? 


o  o  o 


o 


o 


o 


o  o 


Screen  6 
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Section  IV:  Board  Area  Skills 


Screen  7 


Section  V:  Basic  Skills 


Did  UPT  fully  prepare  you  for  follow-on  training  in  IFF  and  your  MWS? 

NOTE:  If  you  were  banked,  consider  your  experience  in  pilot  training  and  not  requal! 


Are  there  any  skills  you  feel  should  be  emphasized  in  or  added  to  UPT? 


Screen  8 


Section  V:  Basic  Skills  (continued) 


List  up  to  5  skills  you  feel  should  be  emphasized  in  or  added  to  UPT 


Screen  9 


Section  VI:  What  to  Add  to  UPT 


Are  there  any  skills/events  taught  in  UPT  you  feel  are  not  required  for  follow-on  training  or  mission  accomplishment? 

□■DH 


Screen  10 
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Section  VI:  What  to  Add  to  U PT 


List  up  to  5  skills/events  taught  in  UPT  you  feel  are  not  required  for  follow-on  training  or  mission  accomplishment? 


Screen  1 1 


Section  VII:  What  to  Delete  from  UPT 


2  Do  you  have  any  further  comments  you  feel  would  assist  in  this  research? 

O  Y  ON 


If  yes,  please  fill  in  the  comment  box  below. 


Screen  12 


Thank-you  for  participating  in  this  research. 

Your  responses  will  have  a  direct  impact  on  the  pipe-line  assignment  process  and 

undergraduate  pilot  training. 

If  you  have  any  questions  about  this  survey, 
please  contact  Maj  Michael  Messer  at  michael.messer@afit 


Screen  13 
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Appendix  B  -  Mini  Survey  #1  -  UPT  Skills  Breakdown 


Appendix  B  is  a  skills  survey  sent  to  instructors  at  ENJJPT.  The  goal  of  the  survey  was 
to  classify  and  assign  maneuvers  flown  in  UPT  into  three  broad  areas  -  Basic,  Instrument,  and 
Mission  Skills. 

Definitions: 

Basic  Skill  -  skills  required  and  applicable  to  any  aircraft 

Instrument  Skill  -  skills  required  to  safely  and  effectively  fly  an  aircraft  in  IMC  and/or  under  IFR 

Mission  Skill  -  a  skill  that  could  be  applied  to  a  specific  mission  or  aircraft  type  such  as  low  level,  air  to  air,  etc 

Instructions:  For  each  task/maneuver,  rank  order  the  three  broad  areas.  If  an  area  does  not  apply,  list  a  "0".  If  you  have  any  comments,  list 
them  under  notes. 

Example: 


Task/Maneuver 

Basic 

Instrument 

Mission  Notes 

1 

Mission  Planning/Briefing/Debriefing 

1 

3 

2 

2 

ILS  Normal 

2 

1 

0 

3 

G-Exercise/Awareness 

2 

0 

1 

1 

Task/Maneuver 

Aerobatic  Maneuvers  (i.e.  loop,  barrel 
roll,  etc) 

Basic  Instrument  Mission  Notes 

2 

Area  Orientation 

3 

Clearing/Visual  Lookout 

4 

Close  Trail 

5 

Communication 

6 

Emergency  Procedures 

7 

Fighting  Wing 

8 

Flight  Integrity/Wingman  Consideration 

9 

Formation  Join-up 

10 

General  Knowledge 

11 

G-Exercise/Awareness 

12  Ground  Operations 

13  Inflight  Planning 

14  Local  Area  Procedures 

15  Map  Preparation 

16  Risk  Management/Decision  Making 

17  Route  Formation 

18  Tactical  Formation 

19  Tactical  Turns 

20  Task  Management 

Are  there  any  other  tasks/maneuvers  you  feel  span  multiple  broad  areas? 
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Appendix  C  -  Mini  Survey  #2  -  Broad  Area  Skill  relationships  based  on  Fighter 

Appendix  C  is  a  follow  up  to  the  survey  found  in  Appendix  B  with  the  goal  of  obtaining 
weightings  between  the  three  broad  areas  based  on  fighter  aircraft  background. 


Instructions:  Select  your  current  fighter  aircraft  or  the  one  you  were  most  recently  qualified  in.  For  each  of  the  four  categories,  select  a  value 
from  1  to  100  indicating  how  important  this  skill  area  is  toward  flying  the  aircraft  you  selected.  The  fou 

Fighter  Aircraft:  (pull  down  menu) 


Basic  Skills 

Instrument  Skills 

Mission  Skills 

Student  Preference 


1(0-100%) 
(0-100%) 
(0-100%) 
(0-100%) 


Basic  Skills  are  those  needed  by  a  pilot  to  fly  any  aircraft  -  fighter  or  otherwise. 

Instrument  Skills  are  those  needed  by  a  pilot  to  fly  an  aircraft  under  Instrument  Flight 
Rules  in  both  VMC  and  IMC. 

Mission  Skills  are  those  needed  by  a  pilot  to  fly  and  execute  a  tactical/combat  mission. 

Student  Preference  acknowledges  a  student  has  a  desire  to  fly  a  particular  aircraft  and  is 
motivated  by  that  desire. 


0%  (Must  =  100%) 
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Appendix  D  -  Mini  Survey  #3  -  UPT  Weighting  Breakdown 

This  is  the  third  survey  sent  to  ENJJPT  instructors.  The  goal  was  to  gather  opinions  and 
weightings  for  the  four  sortie  classifications  and  between  sub-skill  areas  within  the  three  original 
broad  areas.  The  surveys  in  Appendix  B,  C  and  D  were  then  used  to  further  develop  and  refine 
the  FCSS. 


Consider  each  of  the  following  four  areas  (A  through  D).  On  a  scale  of  1  to  10  (10  being  the  greatest  impact/most 
important)  how  would  you  rate  the  contribution  of  each  of  the  following  toward  the  overall  evaluation  of  a  student’s 
performance  and  ability  to  fly  fighter  aircraft? 


A)  T-6/T-37  Daily  rides: 

B)  T-6/T-37  Checkrides: 

C)  T-38  Daily  rides: 

D)  T-38  Checkrides: 

Total: 

Comments: 


Basic  Skills  are  those  needed  by  a  pilot  to  fly  any  aircraft  -  fighter  or  otherwise.  There  are  six  sub-areas  to  consider 
(listed  below).  Each  sub-area  is  further  broken  into  “practical”  categories  that  directly  relate  to  skills  and  maneuvers 
taught  during  pilot  training.  Given  100  points,  distribute  the  points  and  assign  each  sub-area  a  point  value  as  they 
relate  to  each  of  the  other  sub-areas.  If  there  are  other  “sub-areas”  you  feel  apply,  add  the  category  and  point  value. 


General  Knowledge: 

Task  Management: 

Basic  Aircraft  Handling: 

Risk  Management: 

Communication: 

Airmanship: 

Total: 

100  points 

Comments: 
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Instrument  Skills  are  those  needed  by  a  pilot  to  fly  an  aircraft  under  Instrument  Flight  Rules  in  both  VMC  and 
IMC.  There  are  three  sub-areas  to  consider  (listed  below).  Each  sub-area  is  further  broken  into  “practical” 
categories  that  directly  relate  to  skills  and  maneuvers  taught  during  pilot  training.  Given  100  points,  distribute  the 
points  and  assign  each  sub-area  a  point  value  as  they  relate  to  each  of  the  other  sub-areas.  If  there  are  other  “sub- 


areas”  you  feel  apply,  add  the  category  and  point  value. 


Procedures 

Approaches 

Precision  Aircraft  Handling 

Total: 

100  points 

Mission  Skills  are  those  needed  by  a  pilot  to  fly  and  execute  a  tactical/combat  mission.  There  are  five  sub-areas  to 
consider  (listed  below).  Each  sub-area  is  further  broken  into  “practical”  categories  that  directly  relate  to  skills  and 
maneuvers  taught  during  pilot  training.  Given  100  points,  distribute  the  points  and  assign  each  sub-area  a  point 
value  as  they  relate  to  each  of  the  other  sub-areas.  If  there  are  other  “sub-areas”  you  feel  apply,  add  the  category 


and  point  value. 
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Appendix  E  -  Updated  Skills  Survey 

The  original  classification  of  skills  was  broken  into  three  areas  -  Basic,  Instrument  and  Mission 
(with  Formation  added  during  the  working  group).  After  data  analysis,  the  framework  was  modified  to 
General,  Aircraft  Specific,  and  Other  Skills  sets.  The  following  short  survey  was  sent  to  fighter  pilots  at 
AFIT  and  to  instructors  at  ENJJPT  to  obtain  some  data  under  the  new  framework.  A  detailed  definition 
listing  (see  Appendix  G)  and  two  visual  graphic  depictions  of  the  answers  were  provided  to  assist  the 
respondent.  Many  positive  comments  about  the  layout  and  format  were  received  concerning  the  survey 


What  I  need  is  your  opinion  on  the  breakdown  between  the  sub-skill  areas.  Attached  is  a  spreadsheet 
with  two  tabs.  Tab  one  defines  each  skill  area.  If  you  have  any  input,  additions,  or  different  opinions,  add 
these  to  the  third  column.  Tab  two  is  the  actual  input  area  for  the  skill  sets.  For  each  skill  area,  rate  the 
importance  of  the  skill  from  0  (not  important  at  all)  to  1 0  (a  must  have  and  the  most  important  skill  for  a 
pilot  to  have).  The  chart  to  the  right  of  the  input  will  show  the  individual  rating  from  0  to  10.  The  pie  chart 
shows  the  relative  weight  between  each  skill.  If  you  give  each  skill  area  a  10  rating,  the  relative  weights 
between  all  the  skills  in  that  area  will  be  weighted  equally  -  i.e.  you  feel  all  the  skills  under  say  General 
Skills  are  all  equally  important. 

All  three  broad  skill  areas  are  on  the  same  tab  so  you  will  have  to  scroll  down  to  get  to  each  area.  Please 
open  the  file,  add  your  input,  save  and  FORWARD  the  file  to  michael.messer@afit.edu. 


General  Skills 

A/C  Handling 
Basic  Formation 
Crosscheck 
Discipline 

Multi-Tasking/Task  Prioritization 
Risk  Management 
SA 

Sensor  Ops 
Systems  Knowledge 


Value  (0  to  10) 


10 

9 

8 

7 

6 

5 

4 

3 

2 


General  Skills  Comparison 


Sensor  Ops 
SA  6% 


Basic  Formation 
17% 


General  Skills  Comparison 


64 


Value  (0  to  10) 


Aircraft  Specific  Skills 

Advanced  Formation 
Cognitive/Spatial  Thinking 
Communication/Listening 
Crew/Flight  Coordination 
Information/Data  Processing 
Map  Reading 
Mental  Calculations 
Quick  Thinking 
Target  ID 


10 

9 

8 

7 

6 

5 

4 

3 

2 


Aircraft  Specific  Skills  Comparison 

Quick  Thinking 
6% 

Mental 

Calculations 

7% 

Target  II 
4% 

Advanced 

Formation 

18% 

9% 

& 

Coqnitive/Spati 

* 

’  ■ 

17% 

a  Processing 

11% 

Crew/Flight 

Coordination 

13% 

Communication 

/Listening 

15% 

Aircraft  Specific  Skills  Comparison 


Skill  Areas 


Other  Skills 

Attitude 

Compartmentalization 
LOS  Interpretation 
Mission  Planning 
Precision  Flying 


Value  (0  to  10) 


10 

9 

8 

7 

6 


Other  Skills  Comparison 

Precision  Flying 

15% 

Attitude 

24% 

Mission  Planning  ^ - 

LOS 

Interpretation 

20% 

Compartmentaliz 

ation 

23% 

Other  Skills  Comparison 


Skill  Areas 
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Appendix  F  Skills  Survey  Results 

The  following  are  a  summary  of  results  obtained  from  both  the  FCSS  and  other  surveys. 

All  confidence  intervals  are  95%. 

Survey  #s 


Total  #s  of  responses:  A_10  2 

F-15C  65 

F-15E  16 

F-16  19 

F-22  15 

T-37  2 

T-38  3 

Total  FCSS:  122 

Other  Survey  12 

Total  Skills  Surveys:  134 


Original  Broad  Area  breakdown: 


Basic  Skills 

Instrument  Skills 

Mission  Skills 

A-10 

Average 

9.000 

8.500 

10.000 

SD 

1.414 

0.707 

0.000 

Cl: 

0.063 

0.031 

0.000 

F-15C 

Average 

9.062 

7.508 

9.708 

SD 

1.144 

2.070 

0.605 

Cl: 

0.009 

0.016 

0.005 

F-15E 

Average 

8.667 

6.733 

9.667 

SD 

1.047 

1.580 

0.488 

Cl: 

0.016 

0.025 

0.008 

F-16 

Average 

8.947 

7.737 

9.368 

SD 

1.026 

1.821 

0.895 

Cl: 

0.015 

0.026 

0.013 

F-22 

Average 

9.333 

8.133 

9.867 

SD 

0.724 

1.598 

0.352 

Cl: 

0.012 

0.026 

0.006 

T-37 

Average 

7.500 

6.500 

2.000 

SD 

3.536 

3.536 

0.000 

Cl: 

0.157 

0.157 

0.000 

T-38 

Average 

8.333 

8.333 

6.333 

SD 

2.082 

1.528 

3.512 

Cl: 

0.075 

0.055 

0.127 

Overall: 

Average: 

8.984 

7.561 

9.463 

SD: 

1.145 

1.921 

1.332 

Cl: 

0.006 

0.011 

0.008 
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General  Skills  breakdown  by  aircraft  type: 


SA 

Multi- 

Task 

Sensor 

Ops 

Discipline 

Risk 

Mgmt 

A/C 

Handling 

Crosscheck 

Basic 

Form 

Sys  GK 

A-10 

9 

13 

1 

1 

1 

0 

0 

0 

1 

Global: 

15.5% 

24.5% 

1 .7% 

1 .7% 

1 .7% 

0.0% 

0.0% 

0.0% 

1.7% 

Local: 

34.6% 

50.0% 

3.8% 

3.8% 

3.8% 

0.0% 

0.0% 

0.0% 

3.8% 

F-15C 

90 

106 

60 

36 

10 

20 

21 

7 

12 

Global: 

21.5% 

28.1% 

14.4% 

8.6% 

2.4% 

4.8% 

5.0% 

1.7% 

2.9% 

Local: 

24.9% 

29.3% 

16.6% 

9.9% 

2.8% 

5.5% 

5.8% 

1.9% 

3.3% 

F-15E 

36 

57 

33 

3 

6 

0 

9 

0 

3 

Global: 

20.8% 

37.7% 

19.1% 

1 .7% 

3.5% 

0.0% 

5.2% 

0.0% 

1.7% 

Local: 

24.5% 

38.8% 

22.4% 

2.0% 

4.1% 

0.0% 

6.1% 

0.0% 

2.0% 

F-16 

51 

63 

53 

11 

9 

0 

14 

0 

8 

Global: 

20.8% 

28.4% 

21.6% 

4.5% 

3.7% 

0.0% 

5.7% 

0.0% 

3.3% 

Local: 

24.4% 

30.1% 

25.4% 

5.3% 

4.3% 

0.0% 

6.7% 

0.0% 

3.8% 

F-22 

55 

53 

18 

6 

8 

0 

11 

0 

7 

Global: 

29.1% 

32.9% 

9.5% 

3.2% 

4.2% 

0.0% 

5.8% 

0.0% 

3.7% 

Local: 

34.8% 

33.5% 

1 1 .4% 

3.8% 

5.1% 

0.0% 

7.0% 

0.0% 

4.4% 

T-37 

1 

0 

0 

1 

1 

1 

0 

0 

0 

Global: 

20.4% 

0.0% 

0.0% 

20.4% 

20.4% 

20.4% 

0.0% 

0.0% 

0.0% 

Local: 

25.0% 

0.0% 

0.0% 

25.0% 

25.0% 

25.0% 

0.0% 

0.0% 

0.0% 

T-38 

2 

3 

0 

1 

0 

2 

1 

1 

0 

Global: 

14.7% 

25.8% 

0.0% 

7.3% 

0.0% 

14.7% 

7.3% 

7.3% 

0.0% 

Local: 

20.0% 

30.0% 

0.0% 

10.0% 

0.0% 

20.0% 

10.0% 

10.0% 

0.0% 

Other  Survey 

13.6% 

13.1% 

8.9% 

12.2% 

10.0% 

12.8% 

13.3% 

5.6% 

10.5% 

26 

12.5 

9.5 

7 

8.5 

6.5 

2 

3 

Avg: 

20.26% 

21.68% 

10.15% 

10.40% 

8.20% 

9.99% 

9.19% 

3.65% 

6.49% 

SD: 

4.73% 

12.05% 

9.16% 

6.56% 

6.88% 

8.44% 

2.94% 

2.74% 

1 .49% 

Cl: 

0.02% 

0.04% 

0.04% 

0.05% 

0.07% 

0.11% 

0.02% 

0.06% 

0.02% 
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Aircraft  Specific  Skills  breakdown: 


Comm 

Cog/Spatial 

Thinking 

Info/Data 

Proc 

Adv  Form 

Crew 

Coord 

Mental 

Calc 

Quick 

Think 

TgtID 

Map  Read 

A-10 

2 

2 

7 

0 

1 

1 

2 

9 

0 

Global: 

3.4% 

3.4% 

12.1% 

0.0% 

1.7% 

1 .7% 

3.4% 

15.5% 

0.0% 

Local: 

8.3% 

8.3% 

29.2% 

0.0% 

4.2% 

4.2% 

8.3% 

37.5% 

0.0% 

w/other  surv 

12.5% 

8.3% 

21.3% 

1 .7% 

7.9% 

8.8% 

10.0% 

24.6% 

5.0% 

F-15C 

77 

43 

43 

49 

11 

17 

33 

10 

4 

Global: 

20.1% 

10.3% 

10.3% 

11.7% 

2.6% 

4.1% 

7.9% 

2.4% 

1 .0% 

Local: 

26.8% 

15.0% 

15.0% 

17.1% 

3.8% 

5.9% 

1 1 .5% 

3.5% 

1 .4% 

w/other  surv 

20.3% 

15.0% 

13.8% 

15.3% 

7.0% 

7.6% 

10.7% 

5.7% 

4.5% 

F-15E 

23 

21 

16 

11 

25 

4 

7 

26 

3 

Global: 

13.9% 

12.1% 

9.2% 

6.4% 

14.5% 

2.3% 

4.0% 

15.0% 

1 .7% 

Local: 

16.9% 

15.4% 

1 1 .8% 

8.1% 

18.4% 

2.9% 

5.1% 

19.1% 

2.2% 

F-16 

41 

26 

17 

14 

8 

10 

10 

22 

0 

Global: 

18.0% 

10.6% 

6.9% 

5.7% 

3.3% 

4.1% 

4.1% 

9.0% 

0.0% 

Local: 

27.7% 

17.6% 

11.5% 

9.5% 

5.4% 

6.8% 

6.8% 

14.9% 

0.0% 

w/other  surv 

20.2% 

14.8% 

12.1% 

10.8% 

8.2% 

7.9% 

9.2% 

14.0% 

2.9% 

F-22 

7 

20 

33 

0 

2 

7 

16 

0 

0 

Global: 

3.7% 

10.6% 

17.5% 

0.0% 

1.1% 

3.7% 

8.5% 

0.0% 

0.0% 

Local: 

8.2% 

23.5% 

38.8% 

0.0% 

2.4% 

8.2% 

18.8% 

0.0% 

0.0% 

T-37 

1 

0 

0 

1 

2 

0 

0 

0 

0 

Global: 

20.4% 

0.0% 

0.0% 

20.4% 

40.8% 

0.0% 

0.0% 

0.0% 

0.0% 

Local: 

25.0% 

0.0% 

0.0% 

25.0% 

50.0% 

0.0% 

0.0% 

0.0% 

0.0% 

T-38 

1 

2 

0 

0 

1 

1 

2 

0 

1 

Global: 

7.3% 

14.7% 

0.0% 

0.0% 

7.3% 

7.3% 

14.7% 

0.0% 

7.3% 

Local: 

12.5% 

25.0% 

0.0% 

0.0% 

12.5% 

12.5% 

25.0% 

0.0% 

12.5% 

Average: 

17.19% 

15.64% 

12.74% 

9.86% 

16.42% 

6.53% 

1 1 .48% 

6.46% 

3.68% 

SD: 

6.03% 

8.89% 

14.20% 

9.56% 

17.32% 

4.41% 

9.11% 

8.28% 

4.65% 

Cl: 

0.03% 

0.05% 

0.08% 

0.07% 

0.14% 

0.04% 

0.06% 

0.06% 

0.07% 
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Other  Skills  breakdown: 


Prec  Fly 

LOS 

Interp 

Compart  Attitude  MSN  Prep 

A-10 

0 

3 

0 

Global: 

0.0% 

5.2% 

0.0% 

Local: 

0.0% 

100.0% 

0.0% 

F-15C 

8 

1 

6 

Global: 

1 .9% 

0.2% 

1 .4% 

Local: 

53.3% 

6.7% 

40.0% 

F-15E 

0 

1 

1 

Global: 

0.0% 

0.6% 

0.6% 

Local: 

0.0% 

50.0% 

50.0% 

F-16 

5 

5 

1 

Global: 

2.0% 

2.0% 

0.4% 

Local: 

45.5% 

45.5% 

9.1% 

F-22 

0 

0 

1 

Global: 

0.0% 

0.0% 

0.5% 

Local: 

0.0% 

0.0% 

100.0% 

T-37 

0 

0 

0 

Global: 

0.0% 

0.0% 

0.0% 

Local: 

0.0% 

0.0% 

0.0% 

T-38 

0 

0 

0 

Global: 

0.0% 

0.0% 

0.0% 

Local: 

0.0% 

0.0% 

0.0% 

Other  Survey 

15.00% 

18.44% 

20.31% 

27.81% 

18.44% 

Avg: 

1.67% 

1.15% 

0.42% 

1 .28% 

0.85% 

SD: 

0.97% 

1.92% 

0.52% 

Cl: 

0.02% 

0.04% 

0.01% 

Note:  Attitude  and  Mission  Prep  were  not  included  in  the  FCSS.  Because  of  the  lack  of  data 
from  the  FCSS  and  the  potential  incorrect  valuing,  only  the  data  from  the  Update  Skills  survey 
was  used  for  the  Other  Skills  set. 
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Appendix  G  UPT/Skill  Relationships  and  Definitions 

The  following  table  defines  each  sub-skill  area  with  a  relationship  to  one  or  more  UPT 
events.  The  skill  areas  are  classified  by  broad  area.  Note  the  highlighted  areas  are  UPT  events 
not  specifically  or  currently  graded. 


Broad  Area 

Skill  Specific 

UPT  Event 

Definition  and  application  to  skill  area 

General 

SA 

Area  Orientation 

Awareness  of  position  in  the  area 

Fuel  Awareness 

Awareness  of  fuel  state 

G  Awareness 

Awareness  of  aircraft  G  and  relationship  to  max  performance 

In-Flight  Planning 

Adjusting  profile  to  meet  changes  in  surroundings  (weather,  status,  IP 
input,  mission  accomplishment,  fuel  state) 

Pattern  Procedures 

Positional  Awareness  and  following  proper  procedures 

Situation  Recognition 

Does  the  student  recognize  a  dangerous  situation  developing? 

General 

Multi-Tasking/Task  Prioritization 

Basic  Aircraft  Control 

Prioritizations;  aviate,  navigate,  communicate 

Checklist  Usage 

Proper  use  of  checklist  in  all  phases  of  flight 

Composite  Profiles 

Ability  to  complete  a  composite  profile 

Emergency  Procedures 

Proper  application  of  procedures;  solid  understanding  of  procedures 
under  emergent  conditions 

Enroute  Procedures 

Ability  to  prioritize  in-flight  tasks  concerning  mission  profile,  obtaining 
destination  information,  following  established  procedures,  etc 

In  Flight 

Does  the  student  make  proper  use  of  all  resources  available  and/or 
request  additional  assistance  when  needed? 

Instrument  Approaches 

Ability  to  follow  procedures 

Leading  Formation 

Understanding  of  the  multi-aircraft  environment  and  the  decisions 
necessary  to  maintain  control;  ability  to  handle  tasks  beyond  simple 
aircraft  control  in  relation  to  other  aircraft 

Low  Level 

Incorporating  all  inputs  to  successfully  fly  in  LL  environment 

General 

Sensor  Ops 

Cross  Check 

Ability  to  monitor  multiple  inputs  at  once;  inputs  can  be  instruments, 
other  aircraft,  radio,  etc 

Wingman  Consideration 

Aircraft  control  and  consideration  of  external  inputs  affecting  the 
formation  -  sun,  G-on  set,  etc 

General 

Discipline 

Wingman  Procedures 

Adherence  to  contracts  and  established  procedures 

Eps 

Thoroughness  and  handling  of  emergency  procedures 

GK 

Thoroughness  of  knowledge  as  it  relates  the  mission  flown 

Mission  Planning 

Thoroughness  of  completion  of  mission  planning  and  preparation 

Solo  Sorties 

Adherence  to  rules  and  regulations  without  IP  input 

70 


Broad  Area 


Skill  Specific 


UPT  Event 


Definition  and  application  to  skill  area 


General 

Risk  Management 

Area  orientation 

Adjusts  profile  and/or  aircraft  maneuver  to  properly  maintain  area 
boundaries 

Form  landing 

As  lead:  during  landing  phases  assess  the  risk  and  ability  of  the 
formation  to  properly  and  safely  execute  a  formation  landing;  As  wing: 
determines  when  the  risk  level  and  safety  exceed  acceptable  margins 

Formation  Rejoins 

Ability  to  assess  angle  and  closure  and  understands  the  tradeoffs 
between  safety  and  timeliness 

Fuel 

Approaches  profile  management  with  a  consideration  for  fuel  state  and 
maneuvers  remaining 

Go-around 

Recognizes  the  need  to  terminate  a  landing  approach 

In-flight  planning 

Considers  the  risk  posed  by  all  external  and  internal  factors  affecting  the 
flight 

KIO  procedures 

Recognizes  the  need  to  terminate  maneuvering  prior  to  unsafe  condition 
impacting  the  flight 

Overshoots/breakouts 

Recognizes  the  need  to  overshoot  and/or  breakout  during  formation 
maneuvering/rejoin 

General 

A/C  Handling 

Aero 

Aircraft  feel  and  control  to  specific  parameters 

Airmanship 

Aggressiveness  to  be  on  parameters;  confidence 

Echelon  Turn 

Aircraft  feel  and  control  to  specific  parameters 

Fingertip 

Aircraft  feel  and  control  to  specific  parameters 

FM 

Max  performance  of  aircraft 

Instrument  Approaches 

Aircraft  feel  and  control  to  specific  parameters 

Patts/land 

Aircraft  feel  and  control  to  specific  parameters 

Special  Syllabus 

Aircraft  feel  and  control 

Spins 

Aircraft  feel  and  control 

Stalls 

Aircraft  feel  and  control 

General 

Crosscheck 

Area  Orientation 

Monitoring  area  position  in  relation  to  aircraft  movement  and  projected 
course 

Clock-map-ground 

Ability  to  fly  the  aircraft  while  maintaining  certain  parameters  and 
monitoring  the  ground  and  movement  over  the  ground 

Formation  Approach 

Ability  to  fly  aircraft  while  monitoring  other  aircraft 

In  flight  planning 

Monitoring  the  profile  in  relation  to  the  aircraft,  airspace,  fuel  state,  etc 

Instruments 

Uses  a  proper  instrument  crosscheck  to  maintain  a  smooth  platform  for 
instrument  flying 

Over  the  top  aerobatics 

Flying  the  aircraft  within  certain  parameters  on  over-the-top  maneuvers 

Patt/landing 

Ability  to  fly  aircraft  while  monitoring  other  aircraft  and  the  ground  and 
following  proper  procedures 

Tac  Position 

Ability  to  fly  aircraft  while  monitoring  other  aircraft 

Tac  Turn  Contract 

Ability  to  fly  aircraft  to  certain  parameters  while  monitoring  other  aircraft 

Vertical  S 

Use  an  effective  instrument  scan  to  incorporate  all  cockpit  instruments 
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Broad  Area 


Skill  Specific 


UPT  Event 


Definition  and  application  to  skill  area 


General 

Basic  Formation 

Form  T/O  and  Landing 

Safely  execute  Wing  T/O  and  Landing 

Fingertip 

Ability  to  fly  in  close  formation  on  the  wing 

Route 

Ability  to  fly  in  close  formation  on  the  wing 

Close  Trail 

Ability  to  fly  in  close  formation  on  the  wing 

Crossunder 

Ability  to  fly  in  close  formation  on  the  wing 

General 

Systems  Knowledge 

Academics 

Knowledge  and  application  of  information  learned  in  a  classroom  setting 

EPQs 

Knowledge  and  application  of  information  learned  in  a  classroom  setting 

Emergency  Procedures 

Does  the  student  apply  knowledge? 

General  Knowledge 

Knowledge  and  application  of  information  learned  in  a  classroom  setting 

Ground  Ops 

Application  of  systems  knowledge  learned  in  a  classroom  to  pre-flight  of 
the  aircraft;  properly  follows  procedures  and  can  answer  questions 
related  to  aircraft  systems  during  pre-flight 

In  flight 

Ability  for  the  student  to  apply  knowledge  vs  straight  academic  response 

SIMs 

Knowledge  and  application  of  information  learned  in  a  classroom  setting 

Stand-ups 

Knowledge  and  application  of  information  learned  in  a  classroom  setting 
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Broad  Area 

Skill  Specific 

UPT  Event 

Definition  and  application  to  skill  area 

Ability  to  receive  and  translate  information  given  in  the  pre-flight  brief; 

Aircraft  Specific 

Communication/Listening 

Brief/Debrief 

ability  to  reconstruct  and  communicate  information  about  the  flight  in  the 
debrief 

Communication 

Proper  use  of  language  skills  for  the  safe  and  proper  execution  of  the 

mission 

Comm  Out  Procedures 

Proper  use  of  non-verbal  communication  procedures  for  the  safe  and 
proper  conduct  of  the  mission 

Formation 

Proper  use  of  verbal  and  visual  cues  to  control  the  formation  and 
provide  the  necessary  information  to  the  flight  lead  while  in  flight 

Makes  appropriate  inquiries  about  the  mission,  asks  the  necessary 

Mission  Planning 

questions  for  clarification  and  to  obtain  the  necessary  information  to 
conduct  proper  mission  planning 

Out  and  Back 

Effectively  communicates  with  all  the  appropriate  agencies  to  ensure  the 
safely  and  successful  completion  of  the  O+B  mission 

xc 

Effectively  communicates  with  all  the  appropriate  agencies  to  ensure  the 
safely  and  successful  completion  of  the  XC  mission 

Aircraft  Specific 

Cognitive  Thinking 

Area  Planning 

Understands  aircraft  position  in  relation  to  the  area  boundaries  and 
develops  a  plan  for  remaining  within  the  boundaries 

In  Flight  Planning 

Adapts  to  the  in-flight  environment  and  appropriately  develops  a  plan  of 
action  to  complete  the  sortie 

Leading 

Student  has  the  ability  to  forecast  formation  position  and  translate  this 
information  into  appropriate  maneuver  control  and  direction  for  the  flight 

Low  Level  lost  procedures 

Applies  proper  procedures  to  recover  from  a  lost  situation  and  returns  to 
the  known/planned  course 

Risk  Management 

Understands  the  risk  around  the  aircraft/flight  and  develops  an 
appropriate  response  to  the  risk 

Timing 

Ability  to  apply  proper  procedures  to  correct  for  timing  issues  while  in 
flight 

Wind  Correction 

Ability  to  apply  proper  procedures  to  correct  for  wind  issues  while  in 
flight 

Aircraft  Specific 

Information/Data  Processing 

Comm  in  instrument  patt 

Appropriately  processes  information  passed  by  the  ATC  controller  and 
other  aircraft  while  in  the  instrument  pattern 

Fix-to-fix 

Incorporates  all  available  information  to  fly  a  proper  fix-to-fix 

In  Flight 

Ability  to  accurately  interpret  and  analyze  given  information 

Instrument  Approaches 

Incorporates  all  available  information  to  fly  a  proper  instrument  approach 

TOLD 

Appropriately  calculates  and/or  interprets  given  take-off  and  landing 
data;  understands  the  implications  of  TOLD  information 

Aircraft  Specific 

Advanced  Formation 

Tactical  Position 

Attains  and  maintains  a  proper  tactical  formation  position 

Tactical  Rejoins 

Applies  proper  procedures  to  fly  an  expeditious,  safe  tactical  rejoin 

Tactical  Turns 

Applies  proper  geometry  to  turn  at  the  proper  time  to  maintain  position 
following  a  tactical  turn 

Covers  in  appropriate  detail  crew  and  flight  coordination  to  successfully 

Aircraft  Specific 

Crew  Coordination 

Brief/Debrief 

complete  the  mission;  appropriately  identifies  and  addresses 
weaknesses  in  crew/flight  coordination  from  the  flight 

Flight  Discipline 

Maintains  a  proper  level  of  professionalism  in  the  cockpit  while  in  flight 

Flt/Wingman  Coordination 

Considers  the  experience  and  comfort  level  of  all  flight  and  crew 
members  when  conducting  the  mission 

In-flight  Checks 

Conducts  the  appropriate  in-flight  checklist  items  in  a  timely  manner 
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Broad  Area 

Skill  Specific 

UPT  Event 

Definition  and  application  to  skill  area 

Aircraft  Specific 

Mental  Calculations 

Enroute  Descent 

Appropriately  calculates  descent  point 

Enroute  Procedures 

Applies  proper  mental  calculations  and  rules  of  thumb  to  successfully 
complete  the  enroute  portion  of  a  flight 

Fix-to-fix 

Applies  proper  mental  calculations  to  achieve  a  successful  fix-to-fix 

Fuel  balance  checks 

Can  conduct  and  calculate  fuel  balance  checks  in  a  timely  manner 

Ground  Speed  Checks 

Ability  to  calculate  ground  speed  in  a  timely  manner  and  apply  this 
information  to  other  calculations  (timing,  fuel,  etc) 

Holding 

Ability  to  apply  wind  and  timing  correction  factors  using  mental 
calculations 

In-flight  planning 

Ability  to  mentally  perform  necessary  calculations  to  determine  time, 
fuel,  distance,  and  other  factors  affecting  the  flight 

Status  Change 

Able  to  interpret  status  changes  and  incorporate  calculations  into  profile 
changes  and  recalls  as  necessary 

T-38  fuel  comp 

Can  conduct  and  calculate  fuel  balance  checks  in  a  timely  manner 

TOT 

Can  apply  calculations  to  update  speed  to  achieve  a  TOT 

Aircraft  Specific 

Quick  Thinking 

Basic  Formation 

Timely  application  of  control  inputs  to  maintain  formation 

Dynamic  Environment 

Ability  to  adapt  to  a  rapidly  changing  in-flight  environment  and  make 
profile  changes  as  needed 

FM 

Ability  to  translate  visual  cues  into  aircraft  control  inputs  to  achieve 
positional  requirements 

In-flight  Checks  quickly 

Completes  in-flight  checks  in  a  timely  manner  while  maintaining  aircraft 
and  flight  control 

Leading 

Timely  reaction  vs  planned  events;  flexibility 

Rejoins 

Adjusts  aircraft  control  inputs  as  needed  to  safely  and  expeditiously 
achieve  a  rejoin 

Status/fuel  changes 

Ability  to  make  a  timely  decision  concerning  status  and  fuel  state 
changes 

Aircraft  Specific 

Target  ID 

In-Flight  Planning 

Adjusts  aircraft  direction  as  necessary  to  achieve  target  identification 

Maintaining  Course 

Maintains  a  proper  aircraft  heading  to  achieve  target  identification 

Mission  Prep 

Conducts  appropriate  mission  planning  and  review  to  ensure  proper 
target  identification;  utilizes  all  available  resources  to  assist  in  target 
recognition 

Pilotage 

Uses  proper  pilotage  to  achieve  target  identification 

Turn-Point  ID 

Uses  proper  pilotage  to  achieve  turn-point  identification 

Aircraft  Specific 

Map  Reading 

Course  Maintenance 

Properly  interprets  map  data  to  achieve  low  level  course  requirements 

Heading  Control 

Uses  the  map  appropriately  to  maintain  heading  control 

Mission  Planning 

Plans,  completes,  and  interprets  all  available  map/chart  data  to 
successfully  complete  the  mission  (not  just  low  level) 

Pilotage 

Appropriately  interprets  the  map  for  pilotage  to  achieve  successful  low 
level  navigation 

Threat  Awareness 

Is  the  student  able  to  interpret  threats  presented  on  the  map  and  adjust 
the  planning  and/or  mission  profile? 
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Broad  Area 


Skill  Specific 


UPT  Event 


Definition  and  application  to  skill  area 


Other 

Attitude 

Brief/Debrief 

Does  the  student  accept  the  facts  and  not  use  emotion  in  the 
brief/debrief? 

Emotion 

Does  the  student  speak  the  facts  or  use  feelings? 

In  flight 

Is  the  student  an  active  participant  in  flight?  Does  the  student  talk  back 
or  just  acknowledge  without  appropriate  changes  in  behavior? 

Self-Improvement 

Does  the  student  take  previous  lessons  learned  and  make  attempts  to 
improve? 

Other 

Mission  Planning 

Brief/Debrief 

Is  the  student  ready  at  brief/debrief  time,  listen  intently,  respond  when 
queried? 

GK 

Is  the  student  prepared  for  the  mission  in  terms  of  knowledge 
preparation? 

Mission  Materials 

Are  the  mission  materials  correct  and  appropriate  for  the  mission? 
Thoroughness  of  mission  materials 

Preparation 

Has  the  student  utilized  the  appropriate  amount  of  time  preparing  for  the 
sortie? 

Profile 

Has  the  student  developed  an  appropriate  profile  to  meet  training  needs 
and  maximize  training  opportunities? 

Study  Habits 

Does  the  student  utilize  his/her  time  appropriately  when  preparing  and 
studying  for  a  sortie? 

Other 

Precision  Flying 

Approaches 

Accurately  flies  the  instrument  approach  procedures  to  a  minimum 
designated  altitude 

Basic  Control 

Ability  to  maintain  heading,  altitude  and  airspeed  to  established 
parameters 

Close  Trail 

Ability  to  maintain  proper  position 

Fingertip 

Ability  to  maintain  proper  position 

Other 

LOS  Interpretation 

Extended  Trail 

Ability  to  interpret  a  dynamic  changing  environment  and  proper  use  of 
pursuit  curves  to  obtain  and  maintain  a  proper  position 

FM 

Ability  to  interpret  a  dynamic  changing  environment  and  proper  use  of 
pursuit  curves  to  obtain  and  maintain  a  proper  position 

Rejoins 

Ability  to  interpret  a  dynamic  changing  environment  and  proper  use  of 
pursuit  curves  to  obtain  and  maintain  a  proper  position 

Tac  Position 

Ability  to  recognize  deviations  and  positively  correct  back  to  a  proper 
tactical  position 

Tac  Turns 

Ability  to  recognize  line  of  sight  change  and  turn  appropriately  to 
maintain  tactical  position  following  a  turn 

Other 

Compartmentalization 

Discretion 

Does  the  student  temper  his/her  actions? 

IP  to  Target 

Does  the  student  fixate  on  target  or  does  he  consider  his  surroundings 

Mission  Planning 

Ability  to  limit  external  factors  from  impacting  planning  and  prep 

Risk  Management 

Does  the  student  dwell  on  multiple  risk  areas  at  once  or  does  the 
student  consider  the  nearest,  highest  threat  potential? 

Task  Management 

Does  the  student  approach  one  task  at  a  time  or  is  the  student  able  to 
properly  execute  multiple  task  simultaneously? 
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Appendix  H  Student  Skills  Survey 

The  following  survey  was  developed  to  be  sent  to  the  RTU  instructors  to  complete  on 
volunteer  students  to  provide  validation  data. 


For  the  student  listed,  answer  the  following  questions  concerning  the  student’s  current  flying 
ability/skill.  Do  not  consider  any  previous  experience  the  student  may  have  or  any  other  external 
information. 

1)  Advanced  Formation  -  the  ability  to  fly  non-scripted,  tactical  and  advanced  fonnations;  the 
ability  to  make  positive  corrections  to  be  in  position  or  aggressive  return  to  position. 

123456789  10 

Comments: 


2)  Cognitive/Spatial  Thinking  -  the  students  ability  to  think  beyond  his/her  aircraft  and 
consider  not  only  current  events  but  future  affects  and  events  on  not  only  the  aircraft  but 
formation. 

123456789  10 

Comments: 


3)  Communication/Listening  -  hearing,  receiving,  processing  and  understanding  verbal  and 
non-verbal  cues  inside  and  out  of  the  cockpit. 

123456789  10 

Comments: 


4)  Crew/Flight  Coordination  -  Pre-flight,  in-flight,  and  post-flight  coordination  and  conduct 
with  other  crew  members  in  the  aircraft  and  in  the  formation. 

123456789  10 

Comments: 
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5)  Information/Data  Processing  -  ability  to  accurately  interpret  and  analyze  given  information 
(cockpit,  radio,  display,  etc). 

123456789  10 

Comments: 


6)  Map  Reading  -  read  and  interpret  any  and  all  information  presented  in  map/chart  form 
123456789  10 

Comments: 


7)  Mental  Calculations  -  ability  to  accurate  and  quickly  make  computations  on  the  ground  and 
more  specifically,  in  flight. 

123456789  10 

Comments: 


8)  Quick  Thinking  -  proper  decisions  made  without  delay  in  the  mission  profile;  timely  reaction 
vs  planned  events;  ability  to  cope  with  flexibility 

123456789  10 

Comments: 


9)  Target  ID  -  properly  identify  and  distinguish  points  on  the  ground 

123456789  10 

Comments: 
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Appendix  I  ENJJPT  Working  Group  Agenda 


The  ENJJPT  Working  Group  will  refine  the  already  built  value  hierarchy,  skill  sets,  and 
relationships  and  use  the  following  group  goals: 


•  Global  weightings  for  hierarchy 

•  Skill  set  definition 

•  Narrowly  define  measures. 

The  Wing  leadership  will  be  out-briefed  on  the  working  group  results. 
The  following  agenda  will  be  used: 


Time 

Needed 

Timeline 

Item 

Notes 

10  min 

0800 

Overview/VFT  concepts 

Issue/Problem  Statement 

Is  the  system  broke? 

20  min 

0810 

What  makes  a  good  pilot? 

Brainstonning 

10  x  Sticky  pads 

15  min 

0830 

Confirm  Hierarchy 

Broad  area  breakdown 

Flight  Command 
input  issue? 

30  min 

0845 

Refine  UPT  individual  event  weightings 

Poker  chips 

10  min 

0915 

Break 

15  min 

0925 

Relationship  between  T-37/T-38  daily  rides 
and  checkrides 

1000  x  Poker  chips 

50  min 

0940 

Relate  Fighter  Skills  to  UPT  events 

Brainstorm 

10  min 

1030 

Break 

10  min 

1040 

Fighter  A/C  broad  area  weightings 

10  min 

1050 

Summary/Re-cap 

The  working  group  will  consist  of  the  following  ENJJPT  instructors  (prefer  instructors  who  have 
been  flying  >1  year  as  an  IP): 


•  US  SNR 

•  OG  rep 

•  OGT  rep 

•  T-37  FAIP 

•  T-38  FAIP 


Lt  Col  Brad  O’Connor 
Maj  Bruce  Dobbins 
TBD 

Lt  Lucas  Gruenther 
Capt  Josh  Schore 


•  A- 10  background 

•  F-15C  background 

•  F-15E  background 

•  F-16  background 

•  B-l  background 

•  B-52  background 


Capt  Garret  McCoy 
Capt  Jon  Elza 
Maj  George  Truman 
Capt  Peter  Vega 
Capt  Shelby  Bell 
Capt  Chris  Otis 
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